CN110989843A - Image virtual presentation circuit, method, system and wearable device - Google Patents
Image virtual presentation circuit, method, system and wearable device Download PDFInfo
- Publication number
- CN110989843A CN110989843A CN201911284935.6A CN201911284935A CN110989843A CN 110989843 A CN110989843 A CN 110989843A CN 201911284935 A CN201911284935 A CN 201911284935A CN 110989843 A CN110989843 A CN 110989843A
- Authority
- CN
- China
- Prior art keywords
- image
- processed
- current
- brain
- virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Neurosurgery (AREA)
- General Health & Medical Sciences (AREA)
- Neurology (AREA)
- Health & Medical Sciences (AREA)
- Dermatology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
Description
技术领域technical field
本发明涉及图像处理技术领域,尤其涉及一种图像虚拟呈现电路、方法、系统及可穿戴设备。The present invention relates to the technical field of image processing, and in particular, to a circuit, method, system and wearable device for virtual image presentation.
背景技术Background technique
随着互联网和图像处理技术的发展,综艺节目、电影及电视剧等视频也越来越多,人们在工作之余,通常会通过观看视频进行减压、放松,视频对于也越来越融入人们的日常生活之中。With the development of the Internet and image processing technology, there are more and more videos such as variety shows, movies and TV series. People usually watch videos to decompress and relax after work. in daily life.
视频观看者在观看视频时,通常采用的终端设备是智能手机、平板电脑等电子设备,通常视频观看者观看视频的时间比较长,而视频观看者在通过智能手机或平板电脑观看视频时,需要抱着智能手机或平板电脑来观看视频,无法释放双手,需要保持一定的姿势,时间长了身体会疲劳,并且也容易出现视觉疲劳。When video viewers watch videos, the terminal devices usually used are electronic devices such as smartphones and tablet computers. Usually, video viewers watch videos for a long time. Holding a smartphone or tablet to watch a video, you cannot release your hands, you need to maintain a certain posture, your body will get tired after a long time, and you are also prone to visual fatigue.
发明内容SUMMARY OF THE INVENTION
本发明的主要目的在于提供一种图像虚拟呈现电路、方法、系统及可穿戴设备,旨在解决现有技术中用户观看视频时,容易出现身体疲劳和视觉疲劳的技术问题。The main purpose of the present invention is to provide an image virtual presentation circuit, method, system and wearable device, which aims to solve the technical problem of physical fatigue and visual fatigue in the prior art when users watch videos.
为实现上述目的,本发明提供一种图像虚拟呈现电路,所述图像虚拟呈现电路包括:图像处理模块和脑电波传感器;In order to achieve the above object, the present invention provides an image virtual rendering circuit, the image virtual rendering circuit includes: an image processing module and a brain wave sensor;
所述图像处理器,用于获取待处理图像,从所述待处理图像中提取当前图像特征;the image processor, configured to acquire the image to be processed, and extract the current image feature from the image to be processed;
所述图像处理器,还用于通过预设神经网络模型生成与所述当前图像特征对应的当前脑电信号,并将所述当前脑电信号传输至所述脑电波传感器;The image processor is further configured to generate a current EEG signal corresponding to the current image feature through a preset neural network model, and transmit the current EEG signal to the EEG sensor;
所述脑电波传感器,用于发出所述当前脑电信号,以使接收到所述当前脑电信号的用户在大脑中虚拟呈现所述待处理图像。The brain wave sensor is used for sending out the current brain signal, so that the user who receives the current brain signal can virtually present the to-be-processed image in the brain.
可选地,所述图像处理器,还用于获取若干样本图像的图像特征和各样本图像对应的样本脑电信号,通过所述样本图像的图像特征和对应的样本脑电信号对初始神经网络模型进行训练,获得预设神经网络模型。Optionally, the image processor is also used to obtain image features of several sample images and sample EEG signals corresponding to each sample image, and the initial neural network is determined by the image features of the sample images and the corresponding sample EEG signals. The model is trained to obtain a preset neural network model.
可选地,所述图像虚拟呈现电路还包括:通讯模块;Optionally, the image virtual presentation circuit further includes: a communication module;
所述通讯模块,用于接收终端设备传输的待处理图像。The communication module is used for receiving the to-be-processed image transmitted by the terminal device.
可选地,所述图像虚拟呈现电路还包括:音频解码器和扬声器;Optionally, the image virtual presentation circuit further includes: an audio decoder and a speaker;
所述通讯模块,还用于接收终端设备传输的待处理音频;The communication module is also used to receive the audio to be processed transmitted by the terminal device;
所述音频解码器,用于对所述待处理音频进行解码,获得解码结果;The audio decoder is used to decode the audio to be processed to obtain a decoding result;
所述扬声器,用于对所述解码结果进行播放。The speaker is used to play the decoding result.
可选地,所述图像处理器,还用于在从所述待处理图像中提取当前图像特征之前,获取待处理图像的像素点数量;在所述像素点数量超过预设数量时,对所述待处理图像进行压缩。Optionally, the image processor is further configured to acquire the number of pixels of the image to be processed before extracting the current image feature from the image to be processed; when the number of pixels exceeds a preset number, The image to be processed is compressed.
可选地,所述图像处理器,还用于通过K-means算法对所述待处理图像进行压缩。Optionally, the image processor is further configured to compress the to-be-processed image through a K-means algorithm.
此外,为实现上述目的,本发明还提供一种图像虚拟呈现方法,所述图像虚拟呈现方法基于图像虚拟呈现电路实现,所述图像虚拟呈现电路包括:图像处理模块和脑电波传感器;In addition, in order to achieve the above object, the present invention also provides a method for virtual image presentation, the method for virtual image presentation is implemented based on a virtual image presentation circuit, and the virtual image presentation circuit includes: an image processing module and a brain wave sensor;
所述图像虚拟呈现方法包括以下步骤:The image virtual presentation method includes the following steps:
所述图像处理器获取待处理图像,并从所述待处理图像中提取当前图像特征;The image processor acquires the image to be processed, and extracts the current image feature from the image to be processed;
所述图像处理器通过预设神经网络模型生成与所述当前图像特征对应的当前脑电信号,并将所述当前脑电信号传输至所述脑电波传感器;The image processor generates a current EEG signal corresponding to the current image feature through a preset neural network model, and transmits the current EEG signal to the EEG sensor;
所述脑电波传感器发出所述当前脑电信号,以使接收到所述当前脑电信号的用户在大脑中虚拟呈现所述待处理图像。The brain wave sensor sends out the current brain signal, so that the user who receives the current brain signal can virtually present the to-be-processed image in the brain.
此外,为实现上述目的,本发明还提供一种可穿戴设备,所述可穿戴设备为用于设于用户头部的设备,所述可穿戴设备包括所述的图像虚拟呈现电路。In addition, in order to achieve the above object, the present invention also provides a wearable device, the wearable device is a device for setting on the user's head, and the wearable device includes the image virtual presentation circuit.
可选地,所述可穿戴设备为耳机。Optionally, the wearable device is an earphone.
此外,为实现上述目的,本发明还提供一种图像虚拟呈现系统,所述图像虚拟呈现系统包括:终端设备和所述的可穿戴设备,所述终端设备用于将待处理图像传输至所述可穿戴设备。In addition, in order to achieve the above object, the present invention also provides an image virtual presentation system, the image virtual presentation system includes: a terminal device and the wearable device, the terminal device is used to transmit the image to be processed to the Wearable device.
本发明通过图像处理器获取待处理图像,从所述待处理图像中提取当前图像特征,然后图像处理器通过预设神经网络模型生成与所述当前图像特征对应的当前脑电信号,并将所述当前脑电信息传输至所述脑电波传感器,最后通过脑电波传感器发出所述当前脑电信号,以使接收到所述当前脑电信号的用户在大脑中虚拟呈现所述待处理图像,用户不再通过眼睛来获取待处理图像,而是通过大脑虚拟呈现的方式来获取,无需紧盯屏幕,使用户不用长时间拿着电子设备,能够在解放双手的同时,也避免产生视觉疲劳。In the present invention, the image to be processed is acquired by the image processor, the current image feature is extracted from the to-be-processed image, and then the image processor generates the current EEG signal corresponding to the current image feature through a preset neural network model, and then uses the preset neural network model to generate the current EEG signal. The current EEG information is transmitted to the EEG sensor, and finally the current EEG signal is sent out through the EEG sensor, so that the user who receives the current EEG signal can virtually present the image to be processed in the brain. The image to be processed is no longer obtained through the eyes, but obtained through the virtual presentation of the brain. There is no need to stare at the screen, so that the user does not need to hold the electronic device for a long time, which can liberate the hands and avoid visual fatigue.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图示出的结构获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained according to the structures shown in these drawings without creative efforts.
图1为本发明图像虚拟呈现电路一实施例的功能模块图;1 is a functional block diagram of an embodiment of an image virtual presentation circuit of the present invention;
图2为本发明图像虚拟呈现电路二实施例的功能模块图;FIG. 2 is a functional block diagram of a second embodiment of an image virtual presentation circuit according to the present invention;
图3为本发明图像虚拟呈现方法一实施例的流程示意图;FIG. 3 is a schematic flowchart of an embodiment of an image virtual presentation method according to the present invention;
图4为本发明图像虚拟呈现系统一实施例的结构示意图。FIG. 4 is a schematic structural diagram of an embodiment of an image virtual presentation system of the present invention.
附图标号说明:Description of reference numbers:
本发明目的的实现、功能特点及优点将结合实施例,参照附图做进一步说明。The realization, functional characteristics and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments.
具体实施方式Detailed ways
应当理解,此处所描述的具体实施例仅仅用以解释本发明,并不用于限定本发明。It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明的一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
需要说明,本发明实施例中所有方向性指示(诸如上、下、左、右、前、后……)仅用于解释在某一特定姿态(如附图所示)下各部件之间的相对位置关系、运动情况等,如果该特定姿态发生改变时,则该方向性指示也相应地随之改变。It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relationship between various components under a certain posture (as shown in the accompanying drawings). The relative positional relationship, the movement situation, etc., if the specific posture changes, the directional indication also changes accordingly.
另外,在本发明中涉及“第一”、“第二”等的描述仅用于描述目的,而不能理解为指示或暗示其相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。另外,各个实施例之间的技术方案可以相互结合,但是必须是以本领域普通技术人员能够实现为基础,当技术方案的结合出现相互矛盾或无法实现时应当人认为这种技术方案的结合不存在,也不在本发明要求的保护范围之内。In addition, the descriptions involving "first", "second", etc. in the present invention are only for descriptive purposes, and should not be understood as indicating or implying their relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In addition, the technical solutions between the various embodiments can be combined with each other, but must be based on the realization by those of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that the combination of such technical solutions does not exists, and it is not within the protection scope of the present invention.
本发明提出一种图像虚拟呈现电路。The invention provides an image virtual rendering circuit.
参照图1,图1为本发明图像虚拟呈现电路一实施例的功能模块图,在本发明实施例中,所述图像虚拟呈现电路包括:图像处理模块100和脑电波传感器200;其中,Referring to FIG. 1, FIG. 1 is a functional block diagram of an embodiment of an image virtual rendering circuit of the present invention. In the embodiment of the present invention, the image virtual rendering circuit includes: an
所述图像处理器100,用于获取待处理图像,从所述待处理图像中提取当前图像特征。The
需要说明的是,图像中通常存在大量的信息,图像在用户大脑中虚拟呈现类似于用户在大脑中形成回忆画面,回忆画面一般较为抽象,故而,本实施例中只需提取可能呈现在大脑中的部分即可,也就是说,本实施例中可从所述待处理图像中提取对应的当前图像特征。It should be noted that there is usually a large amount of information in the image, and the virtual presentation of the image in the user's brain is similar to that of the user forming a recall picture in the brain. The recall picture is generally abstract. That is, in this embodiment, the corresponding current image feature can be extracted from the to-be-processed image.
可理解的是,所述图像特征可为所述待处理图像的轮廓、色调等信息,本实施例对此不加以限制。It is understandable that, the image feature may be information such as the outline and tone of the to-be-processed image, which is not limited in this embodiment.
例如:所述待处理图像对应某一电视剧在1小时11分的画面,此时,即可从所述待处理图像中提取当前图像特征。For example, the to-be-processed image corresponds to a screen of a TV drama at 1 hour and 11 minutes. At this time, the current image feature can be extracted from the to-be-processed image.
在具体实现中,由于从所述待处理图像中提取当前图像特征时,所述待处理图像的像素点的数量不是固定的,因此,可能会存在待处理图像的像素点数量非常大的情况,但对于本实施例而言,最后的图像是让用户在大脑中虚拟呈现,这个过程类似于用户在大脑中形成回忆画面,即使像素点数量非常高,对用户而言,虚拟呈现的图像也不会随着像素点数量的增加而更加清楚,但像素点数量的增加会延长提取当前图像特征的时长,也会增加处理负荷,故而,本实施例中,所述图像处理器100,还用于在从所述待处理图像中提取当前图像特征之前,获取待处理图像的像素点数量;在所述像素点数量超过预设数量时,对所述待处理图像进行压缩。In a specific implementation, since the number of pixels of the image to be processed is not fixed when the current image feature is extracted from the image to be processed, there may be a situation where the number of pixels of the image to be processed is very large, However, for this embodiment, the final image is to be virtually presented in the user's brain. This process is similar to the user forming a memory image in the brain. Even if the number of pixels is very high, for the user, the virtual presented image is not It will become clearer with the increase of the number of pixels, but the increase of the number of pixels will prolong the time for extracting the current image features, and also increase the processing load. Therefore, in this embodiment, the
假设所述预设数量为640*480,若数字信号的像素点数量为1600*1200,此时,需要对所述数字信号进行压缩。Assuming that the preset number is 640*480, if the number of pixels of the digital signal is 1600*1200, at this time, the digital signal needs to be compressed.
具体地,在对所述数字信号进行压缩时,可通过均值压缩来进行压缩,也就是说,将数字信号中的每N*N作为一个对象,对每个对象中的像素的颜色值进行均值计算,并将均值计算结果作为该对象的颜色值,最后将每个对象作为一个像素,此时,可将数字信号压缩N*N倍,假设N*N为4*4,压缩后的数字信号的像素点数量为400*300。Specifically, when compressing the digital signal, the compression can be performed by means of mean value compression, that is, taking every N*N in the digital signal as an object, and averaging the color values of the pixels in each object Calculate and use the mean calculation result as the color value of the object, and finally use each object as a pixel. At this time, the digital signal can be compressed N*N times, assuming that N*N is 4*4, the compressed digital signal The number of pixels is 400*300.
当然,为了保证压缩后的图像保真率,本实施例中,所述图像处理器100,还用于通过K-means算法对所述待处理图像进行压缩。Of course, in order to ensure the fidelity of the compressed image, in this embodiment, the
所述图像处理器100,还用于通过预设神经网络模型生成与所述当前图像特征对应的当前脑电信号,并将所述当前脑电信号传输至所述脑电波传感器。The
在具体实现中,近期在现有技术中出现了一种根据脑电波重现人眼看到的图像的技术,也就是说,在大脑的脑电波中存在大量信息,根据脑电波中的信息能够重现人眼看到的图像,故而,本实施例中,可预先建立一个预设神经网络模型,所述预设神经网络模型能够反映脑电信号和图像特征之间的对应关系,因此,可通过预设神经网络模型生成与所述当前图像特征对应的当前脑电信号,并将所述当前脑电信号传输至所述脑电波传感器200。In the specific implementation, a technology for reproducing images seen by the human eye based on brain waves has recently appeared in the prior art. Therefore, in this embodiment, a preset neural network model can be established in advance, and the preset neural network model can reflect the corresponding relationship between EEG signals and image features. It is assumed that the neural network model generates a current EEG signal corresponding to the current image feature, and transmits the current EEG signal to the
为便于生成预设神经网络模型,本实施例中,所述图像处理器100,还用于获取若干样本图像的图像特征和各样本图像对应的样本脑电信号,通过所述样本图像的图像特征和对应的样本脑电信号对初始神经网络模型进行训练,获得预设神经网络模型。In order to facilitate the generation of the preset neural network model, in this embodiment, the
可理解的是,为保证样本图像的图像特征和样本脑电信号之间的对应关系,本实施例中,对于所述样本图像对应的样本脑电信号而言,其可为观看样本图像时产生的脑电信号,也就是说,可以在观看样本图像时,收集脑电信号,并将收集的脑电信号作为所述样本图像对应的样本脑电信号。It is understandable that, in order to ensure the correspondence between the image features of the sample image and the sample EEG signal, in this embodiment, for the sample EEG signal corresponding to the sample image, it may be generated when viewing the sample image. The EEG signal, that is, the EEG signal can be collected when viewing the sample image, and the collected EEG signal can be used as the sample EEG signal corresponding to the sample image.
由于脑电信号通常会在一定频率范围内,也就是说,人的大脑能够接收到的脑电信号具有一定的范围,通常来说,脑电信号可划分为四个波段,即δ(1-3Hz)、θ(4-7Hz)、α(8-13Hz)和β(14-30Hz)。其中,δ波,频率为每秒1-3次,当人在婴儿期或智力发育不成熟、成年人在极度疲劳和昏睡状态下,可出现这种波段。θ波,频率为每秒4-7次,成年人在意愿受到挫折和抑郁时以及精神病患者这种波极为显著。但此波为少年(10-17岁)的脑电图中的主要成分。α波,频率为每秒8-13次,平均数为10次左右,它是正常人脑电波的基本节律,如果没有外加的刺激,其频率是相当恒定的。人在清醒、安静并闭眼时该节律最为明显,睁开眼睛或接受其它刺激时,α波即刻消失。β波,频率为每秒14-30次,当精神紧张和情绪激动或亢奋时出现此波,当人从睡梦中惊醒时,原来的慢波节律可立即被该节律所替代。Since the EEG signal is usually within a certain frequency range, that is to say, the EEG signal that the human brain can receive has a certain range. Generally speaking, the EEG signal can be divided into four bands, namely δ(1- 3Hz), θ (4-7Hz), α (8-13Hz) and β (14-30Hz). Among them, the delta wave, the frequency is 1-3 times per second, when people are in infancy or immature intellectual development, adults are extremely fatigued and lethargic, this wave band can appear. Theta waves, with a frequency of 4-7 times per second, are extremely pronounced in adults when their will is frustrated and depressed, as well as in mentally ill patients. However, this wave is the main component in the EEG of teenagers (10-17 years old). Alpha wave, the frequency is 8-13 times per second, the average number is about 10 times, it is the basic rhythm of normal human brain waves, if there is no external stimulus, its frequency is quite constant. The rhythm is most obvious when a person is awake, quiet, and eyes closed, and the alpha wave disappears immediately when eyes are opened or when they receive other stimuli. Beta wave, the frequency is 14-30 times per second, this wave appears when mental tension and emotional excitement or excitement, when people wake up from sleep, the original slow wave rhythm can be replaced by this rhythm immediately.
在具体实现中,由于不同用户之间的脑电信号可能存在个体差异,为了保证生成的脑电信号能够符合用户的脑电信号,本实施例中,可在用户观看样本图像时,收集用户的脑电信号,并将收集的脑电信号作为所述样本图像对应的样本脑电信号,也就是说,对于不同用户而言,分别对应有不同的预设神经网络模型。In a specific implementation, since there may be individual differences in the EEG signals between different users, in order to ensure that the generated EEG signals can conform to the user's EEG signals, in this embodiment, the user's EEG signals can be collected when the user watches the sample image. EEG signals, and the collected EEG signals are used as the sample EEG signals corresponding to the sample images, that is, for different users, there are different preset neural network models respectively.
所述脑电波传感器200,用于发出所述当前脑电信号,以使接收到所述当前脑电信号的用户在大脑中虚拟呈现所述待处理图像。The
本实施例通过图像处理器获取待处理图像,从所述待处理图像中提取当前图像特征,然后图像处理器通过预设神经网络模型生成与所述当前图像特征对应的当前脑电信号,并将所述当前脑电信息传输至所述脑电波传感器,最后通过脑电波传感器发出所述当前脑电信号,以使接收到所述当前脑电信号的用户在大脑中虚拟呈现所述待处理图像,用户不再通过眼睛来获取待处理图像,而是通过大脑虚拟呈现的方式来获取,无需紧盯屏幕,使用户不用长时间拿着电子设备,能够在解放双手的同时,也避免产生视觉疲劳。In this embodiment, an image to be processed is acquired by an image processor, and current image features are extracted from the to-be-processed image, and then the image processor generates a current EEG signal corresponding to the current image feature through a preset neural network model, and uses The current EEG information is transmitted to the EEG sensor, and finally the current EEG signal is sent out through the EEG sensor, so that the user who receives the current EEG signal can virtually present the image to be processed in the brain, The user no longer obtains the image to be processed through the eyes, but obtains it through the virtual presentation of the brain. There is no need to stare at the screen, so that the user does not need to hold the electronic device for a long time. While freeing hands, it also avoids visual fatigue.
进一步地,如图2所示,基于第一实施例提出本发明图像虚拟呈现电路第二实施例。Further, as shown in FIG. 2 , based on the first embodiment, a second embodiment of the image virtual rendering circuit of the present invention is proposed.
在本实施例中,所述图像虚拟呈现电路还包括:通讯模块300;In this embodiment, the image virtual presentation circuit further includes: a
所述通讯模块300,用于接收终端设备传输的待处理图像。The
需要说明的是,所述终端设备为可用于对视频进行处理的设备,例如:笔记本电脑、个人电脑、平板电脑或智能手机等设备,本实施例对此不加以限制。It should be noted that the terminal device is a device that can be used to process video, such as a notebook computer, a personal computer, a tablet computer, or a smart phone, which is not limited in this embodiment.
在具体实现中,所述通讯模块300可为无线通讯模块,也就是说,其可先与所述终端设备进行无线配对,以实现无线通讯连接,从而可通过蓝牙通信协议来接收终端设备传输的待处理图像,当然,也可通过3G、4G或5G通讯协议来接收终端设备传输的待处理图像,本实施例对此不加以限制。In a specific implementation, the
假设所述图像虚拟呈现电路处于蓝牙耳机中,此时,通讯模块300通过无线蓝牙与终端设备进行配对后,可通过5G通讯协议接收终端设备传输的待处理图像。Assuming that the image virtual presentation circuit is in the Bluetooth headset, at this time, after the
当然,所述通讯模块300也可为有线通讯模块,也就是说,其可为一个通讯接口,通过通讯接口来接收终端设备传输的待处理图像。Of course, the
进一步地,在本实施例中,所述图像虚拟呈现电路还包括:音频解码器(未示出)和扬声器(未示出)。Further, in this embodiment, the image virtual presentation circuit further includes: an audio decoder (not shown) and a speaker (not shown).
所述通讯模块300,还用于接收终端设备传输的待处理音频。The
所述音频解码器,用于对所述待处理音频进行解码,获得解码结果。The audio decoder is configured to decode the to-be-processed audio to obtain a decoding result.
所述扬声器,用于对所述解码结果进行播放。The speaker is used to play the decoding result.
需要说明的是,通常而言,在大脑中虚拟呈现所述待处理图像是一个持续的过程,但假设时间过长会显得非常单调,为了保证用户能够同时接收到声音和图像,从而丰富用户接收到的信息种类,本实施例中,所述通讯模块300接收终端设备传输的待处理音频,所述音频解码器对所述待处理音频进行解码,获得解码结果,所述扬声器对所述解码结果进行播放。It should be noted that, generally speaking, the virtual presentation of the image to be processed in the brain is a continuous process, but if the time is too long, it will appear very monotonous. In this embodiment, the
同样假设所述图像虚拟呈现电路处于蓝牙耳机中,当终端设备中在播放某一电影时,此时,可将终端设备中所播放的内容(图像和音频)传输到蓝牙耳机上进行信号处理。It is also assumed that the virtual image presentation circuit is in a Bluetooth headset. When a movie is being played in the terminal device, the content (image and audio) played in the terminal device can be transmitted to the Bluetooth headset for signal processing.
此外,为实现上述目的,本发明实施例还提出一种图像虚拟呈现方法,参照图3,图3为本发明图像虚拟呈现方法第一实施例的流程示意图。In addition, to achieve the above purpose, an embodiment of the present invention further provides a method for virtual image presentation. Referring to FIG. 3 , FIG. 3 is a schematic flowchart of the first embodiment of the method for virtual image presentation of the present invention.
在第一实施例中,所述图像虚拟呈现方法基于图像虚拟呈现电路实现,所述图像虚拟呈现电路包括:图像处理模块和脑电波传感器;In the first embodiment, the image virtual rendering method is implemented based on an image virtual rendering circuit, and the image virtual rendering circuit includes: an image processing module and a brain wave sensor;
所述图像虚拟呈现方法包括以下步骤:The image virtual presentation method includes the following steps:
S10:所述图像处理器获取待处理图像,并从所述待处理图像中提取当前图像特征。S10: The image processor acquires the to-be-processed image, and extracts the current image feature from the to-be-processed image.
需要说明的是,图像中通常存在大量的信息,图像在用户大脑中虚拟呈现类似于用户在大脑中形成回忆画面,回忆画面一般较为抽象,故而,本实施例中只需提取可能呈现在大脑中的部分即可,也就是说,本实施例中可从所述待处理图像中提取对应的当前图像特征。It should be noted that there is usually a large amount of information in the image, and the virtual presentation of the image in the user's brain is similar to that of the user forming a recall picture in the brain. The recall picture is generally abstract. That is, in this embodiment, the corresponding current image feature can be extracted from the to-be-processed image.
可理解的是,所述图像特征可为所述待处理图像的轮廓、色调等信息,本实施例对此不加以限制。It is understandable that, the image feature may be information such as the outline and tone of the to-be-processed image, which is not limited in this embodiment.
例如:所述待处理图像对应某一电视剧在1小时11分的画面,此时,即可从所述待处理图像中提取当前图像特征。For example, the to-be-processed image corresponds to a screen of a TV drama at 1 hour and 11 minutes. At this time, the current image feature can be extracted from the to-be-processed image.
在具体实现中,由于从所述待处理图像中提取当前图像特征时,所述待处理图像的像素点的数量不是固定的,因此,可能会存在待处理图像的像素点数量非常大的情况,但对于本实施例而言,最后的图像是让用户在大脑中虚拟呈现,这个过程类似于用户在大脑中形成回忆画面,即使像素点数量非常高,对用户而言,虚拟呈现的图像也不会随着像素点数量的增加而更加清楚,但像素点数量的增加会延长提取当前图像特征的时长,也会增加处理负荷,故而,本实施例中,所述图像处理器100,还用于在从所述待处理图像中提取当前图像特征之前,获取待处理图像的像素点数量;在所述像素点数量超过预设数量时,对所述待处理图像进行压缩。In a specific implementation, since the number of pixels of the image to be processed is not fixed when the current image feature is extracted from the image to be processed, there may be a situation where the number of pixels of the image to be processed is very large, However, for this embodiment, the final image is to be virtually presented in the user's brain. This process is similar to the user forming a memory image in the brain. Even if the number of pixels is very high, for the user, the virtual presented image is not It will become clearer with the increase of the number of pixels, but the increase of the number of pixels will prolong the time for extracting the current image features, and also increase the processing load. Therefore, in this embodiment, the
假设所述预设数量为640*480,若数字信号的像素点数量为1600*1200,此时,需要对所述数字信号进行压缩。Assuming that the preset number is 640*480, if the number of pixels of the digital signal is 1600*1200, at this time, the digital signal needs to be compressed.
具体地,在对所述数字信号进行压缩时,可通过均值压缩来进行压缩,也就是说,将数字信号中的每N*N作为一个对象,对每个对象中的像素的颜色值进行均值计算,并将均值计算结果作为该对象的颜色值,最后将每个对象作为一个像素,此时,可将数字信号压缩N*N倍,假设N*N为4*4,压缩后的数字信号的像素点数量为400*300。Specifically, when compressing the digital signal, the compression can be performed by means of mean value compression, that is, taking every N*N in the digital signal as an object, and averaging the color values of the pixels in each object Calculate and use the mean calculation result as the color value of the object, and finally use each object as a pixel. At this time, the digital signal can be compressed N*N times, assuming that N*N is 4*4, the compressed digital signal The number of pixels is 400*300.
当然,为了保证压缩后的图像保真率,本实施例中,所述图像处理器100,还用于通过K-means算法对所述待处理图像进行压缩。Of course, in order to ensure the fidelity of the compressed image, in this embodiment, the
S20:所述图像处理器通过预设神经网络模型生成与所述当前图像特征对应的当前脑电信号,并将所述当前脑电信号传输至所述脑电波传感器。S20: The image processor generates a current EEG signal corresponding to the current image feature by using a preset neural network model, and transmits the current EEG signal to the EEG sensor.
在具体实现中,近期在现有技术中出现了一种根据脑电波重现人眼看到的图像的技术,也就是说,在大脑的脑电波中存在大量信息,根据脑电波中的信息能够重现人眼看到的图像,故而,本实施例中,可预先建立一个预设神经网络模型,所述预设神经网络模型能够反映脑电信号和图像特征之间的对应关系,因此,可通过预设神经网络模型生成与所述当前图像特征对应的当前脑电信号,并将所述当前脑电信号传输至所述脑电波传感器200。In the specific implementation, a technology for reproducing images seen by the human eye based on brain waves has recently appeared in the prior art. Therefore, in this embodiment, a preset neural network model can be established in advance, and the preset neural network model can reflect the corresponding relationship between EEG signals and image features. It is assumed that the neural network model generates a current EEG signal corresponding to the current image feature, and transmits the current EEG signal to the
为便于生成预设神经网络模型,本实施例中,所述图像处理器100,还用于获取若干样本图像的图像特征和各样本图像对应的样本脑电信号,通过所述样本图像的图像特征和对应的样本脑电信号对初始神经网络模型进行训练,获得预设神经网络模型。In order to facilitate the generation of the preset neural network model, in this embodiment, the
可理解的是,为保证样本图像的图像特征和样本脑电信号之间的对应关系,本实施例中,对于所述样本图像对应的样本脑电信号而言,其可为观看样本图像时产生的脑电信号,也就是说,可以在观看样本图像时,收集脑电信号,并将收集的脑电信号作为所述样本图像对应的样本脑电信号。It is understandable that, in order to ensure the correspondence between the image features of the sample image and the sample EEG signal, in this embodiment, for the sample EEG signal corresponding to the sample image, it may be generated when viewing the sample image. The EEG signal, that is, the EEG signal can be collected when viewing the sample image, and the collected EEG signal can be used as the sample EEG signal corresponding to the sample image.
由于脑电信号通常会在一定频率范围内,也就是说,人的大脑能够接收到的脑电信号具有一定的范围,通常来说,脑电信号可划分为四个波段,即δ(1-3Hz)、θ(4-7Hz)、α(8-13Hz)和β(14-30Hz)。其中,δ波,频率为每秒1-3次,当人在婴儿期或智力发育不成熟、成年人在极度疲劳和昏睡状态下,可出现这种波段。θ波,频率为每秒4-7次,成年人在意愿受到挫折和抑郁时以及精神病患者这种波极为显著。但此波为少年(10-17岁)的脑电图中的主要成分。α波,频率为每秒8-13次,平均数为10次左右,它是正常人脑电波的基本节律,如果没有外加的刺激,其频率是相当恒定的。人在清醒、安静并闭眼时该节律最为明显,睁开眼睛或接受其它刺激时,α波即刻消失。β波,频率为每秒14-30次,当精神紧张和情绪激动或亢奋时出现此波,当人从睡梦中惊醒时,原来的慢波节律可立即被该节律所替代。Since the EEG signal is usually within a certain frequency range, that is to say, the EEG signal that the human brain can receive has a certain range. Generally speaking, the EEG signal can be divided into four bands, namely δ(1- 3Hz), θ (4-7Hz), α (8-13Hz) and β (14-30Hz). Among them, the delta wave, the frequency is 1-3 times per second, when people are in infancy or immature intellectual development, adults are extremely fatigued and lethargic, this wave band can appear. Theta waves, with a frequency of 4-7 times per second, are extremely prominent in adults when they are frustrated and depressed, and in mentally ill patients. However, this wave is the main component in the EEG of teenagers (10-17 years old). Alpha wave, the frequency is 8-13 times per second, the average number is about 10 times, it is the basic rhythm of normal human brain waves, if there is no external stimulus, its frequency is quite constant. The rhythm is most obvious when a person is awake, quiet, and eyes closed, and the alpha wave disappears immediately when eyes are opened or when they receive other stimuli. Beta wave, the frequency is 14-30 times per second, this wave appears when mental tension and emotional excitement or excitement, when people wake up from sleep, the original slow wave rhythm can be replaced by this rhythm immediately.
在具体实现中,由于不同用户之间的脑电信号可能存在个体差异,为了保证生成的脑电信号能够符合用户的脑电信号,本实施例中,可在用户观看样本图像时,收集用户的脑电信号,并将收集的脑电信号作为所述样本图像对应的样本脑电信号,也就是说,对于不同用户而言,分别对应有不同的预设神经网络模型。In a specific implementation, since there may be individual differences in the EEG signals between different users, in order to ensure that the generated EEG signals can conform to the user's EEG signals, in this embodiment, the user's EEG signals can be collected when the user watches the sample image. EEG signals, and the collected EEG signals are used as the sample EEG signals corresponding to the sample images, that is, for different users, there are different preset neural network models respectively.
S30:所述脑电波传感器发出所述当前脑电信号,以使接收到所述当前脑电信号的用户在大脑中虚拟呈现所述待处理图像。S30: The brain wave sensor sends the current brain signal, so that the user who receives the current brain signal can virtually present the to-be-processed image in the brain.
本实施例通过图像处理器获取待处理图像,从所述待处理图像中提取当前图像特征,然后图像处理器通过预设神经网络模型生成与所述当前图像特征对应的当前脑电信号,并将所述当前脑电信息传输至所述脑电波传感器,最后通过脑电波传感器发出所述当前脑电信号,以使接收到所述当前脑电信号的用户在大脑中虚拟呈现所述待处理图像,用户不再通过眼睛来获取待处理图像,而是通过大脑虚拟呈现的方式来获取,无需紧盯屏幕,使用户不用长时间拿着电子设备,能够在解放双手的同时,也避免产生视觉疲劳。In this embodiment, an image to be processed is acquired by an image processor, and current image features are extracted from the to-be-processed image, and then the image processor generates a current EEG signal corresponding to the current image feature through a preset neural network model, and uses The current EEG information is transmitted to the EEG sensor, and finally the current EEG signal is sent out through the EEG sensor, so that the user who receives the current EEG signal can virtually present the image to be processed in the brain, The user no longer obtains the image to be processed through the eyes, but obtains it through the virtual presentation of the brain. There is no need to stare at the screen, so that the user does not need to hold the electronic device for a long time. While freeing hands, it also avoids visual fatigue.
本实施例的方法还能用于实现上述图像虚拟呈现电路中的功能,在此不再赘述。The method in this embodiment can also be used to implement the functions in the above-mentioned image virtual rendering circuit, which will not be repeated here.
此外,为实现上述目的,本发明实施例还提出一种可穿戴设备,所述可穿戴设备为用于设于用户头部的设备,所述可穿戴设备包括如上所述的图像虚拟呈现电路。In addition, in order to achieve the above object, an embodiment of the present invention further provides a wearable device, the wearable device is a device for setting on the user's head, and the wearable device includes the above-mentioned virtual image presentation circuit.
需要说明的是,对于所述可穿戴设备而言,其只要为可设于用户头部的设备即可,例如:蓝牙耳机、脑电帽、智能眼镜等设备,本实施例对此不加以限制。It should be noted that, for the wearable device, it only needs to be a device that can be installed on the user's head, such as a Bluetooth headset, an EEG cap, smart glasses and other devices, which are not limited in this embodiment. .
此外,为实现上述目的,本发明实施例还提出一种图像虚拟呈现系统,所述图像虚拟呈现系统包括:终端设备和如上所述的可穿戴设备。In addition, in order to achieve the above object, an embodiment of the present invention also provides a virtual image presentation system, where the virtual image presentation system includes: a terminal device and the above-mentioned wearable device.
可理解的是,在所述可穿戴设备为蓝牙耳机时,由于其具有成本低、重量轻等优点,因此,便于用户进行使用,具体可参照图4,其中,所述通讯模块11及通讯模块12可以内置于蓝牙耳机1中,无线通讯模块21内置在终端设备2中,该蓝牙耳机1上的通讯模块11及通讯模块12分别内置在左边的蓝牙耳机和右边的蓝牙耳机上,通讯模块11及通讯模块12可与终端设备2上的无线通讯模块21之间进行无线配对实现无线通讯连接,使得蓝牙耳机设备1可通过该无线通讯连接从而实现接收终端设备2中的待处理图像。通讯模块11及通讯模块12与无线通讯模块21可以基于目前的5G通讯协议实现视频信号通讯。例如:蓝牙耳机1与终端设备2通过无线蓝牙进行配对,配对成功后,可通过5G通讯协议将终端设备2里所播放的内容(图像和音频)传输到蓝牙耳机1上进行信号处理。It can be understood that when the wearable device is a Bluetooth headset, because of its advantages of low cost and light weight, it is convenient for users to use. 12 can be built in the
在通过终端设备2将待处理图像发送至蓝牙耳机1中后,蓝牙耳机设备1里的图像处理器对所述待处理图像进行处理后,获得数字信号,再通过蓝牙耳机1中的脑电波传感器13和脑电波传感器14将数字信号转换成用户的大脑能够接受的频率信号,在用于大脑里呈现手机中的图像。由于蓝牙耳机1是贴人皮肤佩戴的,脑电波传感器13和脑电波传感器14可以接受到人脑的脑电波的频率,再可通过脑电波传感器将数字信号传换成用户大脑的脑电波的频率范围,这样大脑就能够接受到终端设备2所播放的图像。After the image to be processed is sent to the
对于所述蓝牙耳机1的供电部分可以与无线通讯过程相似,可以直接通过终端设备2向蓝牙耳机1进行供电,使得终端设备2中的电源模块22的电能可供给蓝牙耳机1,或者终端设备2可以直接对蓝牙耳机1的电源模块15和电源模块16进行无线充电。当然也可以直接通过电源模块15和电源模块16直接给蓝牙耳机1供电,本实施例对此不加以限制。The power supply part of the
当然,蓝牙耳机1可以音频和视频直接切换使用,在终端设备2中,如果只播放音频,蓝牙耳机1中将无图像信号处理,此时蓝牙耳机1起到播放音乐的功能,只能听音乐;如果终端设备2在播放视频(如MV、电视剧、电影等)时,蓝牙耳机1的图像处理模块会工作,此时即可以虚拟呈现图像,也可以听到视频中的声音,故而,用户可以根据需要自由切换。Of course,
以上仅为本发明的优选实施例,并非因此限制本发明的专利范围,凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换,或直接或间接运用在其他相关的技术领域,均同理包括在本发明的专利保护范围内。The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911284935.6A CN110989843B (en) | 2019-12-13 | 2019-12-13 | Image virtual presentation circuit, method and system and wearable device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911284935.6A CN110989843B (en) | 2019-12-13 | 2019-12-13 | Image virtual presentation circuit, method and system and wearable device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110989843A true CN110989843A (en) | 2020-04-10 |
| CN110989843B CN110989843B (en) | 2024-02-09 |
Family
ID=70093504
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911284935.6A Active CN110989843B (en) | 2019-12-13 | 2019-12-13 | Image virtual presentation circuit, method and system and wearable device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110989843B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105630145A (en) * | 2015-03-28 | 2016-06-01 | 朱金彪 | Virtual sense realization method and apparatus as well as glasses or helmet using same |
| CN109419582A (en) * | 2017-08-27 | 2019-03-05 | 南京乐朋电子科技有限公司 | A kind of novel visual converyer |
| CN109697831A (en) * | 2019-02-25 | 2019-04-30 | 湖北亿咖通科技有限公司 | Fatigue driving monitoring method, device and computer readable storage medium |
| CN110333786A (en) * | 2019-07-16 | 2019-10-15 | 李蕴梅 | It is a kind of to obtain the brain machine interconnected method that brain signal is used to store, transmit and convert by external equipment |
-
2019
- 2019-12-13 CN CN201911284935.6A patent/CN110989843B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105630145A (en) * | 2015-03-28 | 2016-06-01 | 朱金彪 | Virtual sense realization method and apparatus as well as glasses or helmet using same |
| CN109419582A (en) * | 2017-08-27 | 2019-03-05 | 南京乐朋电子科技有限公司 | A kind of novel visual converyer |
| CN109697831A (en) * | 2019-02-25 | 2019-04-30 | 湖北亿咖通科技有限公司 | Fatigue driving monitoring method, device and computer readable storage medium |
| CN110333786A (en) * | 2019-07-16 | 2019-10-15 | 李蕴梅 | It is a kind of to obtain the brain machine interconnected method that brain signal is used to store, transmit and convert by external equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110989843B (en) | 2024-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101467875B (en) | Ear-worn Physiological Feedback Devices | |
| CN107683399B (en) | Sound output device, electronic device, and control method thereof | |
| CN107049280B (en) | Wearable equipment of mobile internet intelligence | |
| KR20220093306A (en) | Electronic device and method for measuring biometric signals | |
| US20190240842A1 (en) | Robot assisted interaction system and method thereof | |
| CN108491076B (en) | Display control method and related products | |
| CN205862013U (en) | A kind of light-duty VR eyesight-protecting glasses system | |
| EP4571756A1 (en) | Electroencephalogram analysis model training method and apparatus, computer device, computer-readable storage medium and computer program product | |
| JP7701368B2 (en) | Emotion Determination Device, Emotion Determination Method, and Emotion Determination Program | |
| CN105630145A (en) | Virtual sense realization method and apparatus as well as glasses or helmet using same | |
| CN108874130A (en) | Control method for playing back and Related product | |
| US11368664B2 (en) | Information processing apparatus, information processing method, and program | |
| US20200073627A1 (en) | Method of streaming motion picture | |
| JP2002166050A (en) | Method of computer game using bio feed back of brain wave and its apparatus | |
| CN108418962B (en) | Information response method based on brain wave and related product | |
| CN108304074A (en) | Display control method and related product | |
| KR102729208B1 (en) | Electronic apparatus and method of inducing sleep | |
| CN110236537B (en) | VR meditation control method, equipment and system based on brain wave detection | |
| CN110989843B (en) | Image virtual presentation circuit, method and system and wearable device | |
| CN108334200B (en) | Electronic equipment control method and related products | |
| CN108418959A (en) | Electronic device, method for outputting prompt information and related product | |
| CN108144291A (en) | Brain wave-based game control method and related products | |
| CN108829244B (en) | Volume adjusting method and related equipment | |
| TW201617104A (en) | Auxiliary system and method for aiding sleep | |
| CN108594993B (en) | Electronic device, audio playback method and related products |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201016 Address after: 261031, north of Jade East Street, Dongming Road, Weifang hi tech Zone, Shandong province (GoerTek electronic office building, Room 502) Applicant after: GoerTek Optical Technology Co.,Ltd. Address before: 261031 Dongfang Road, Weifang high tech Industrial Development Zone, Shandong, China, No. 268 Applicant before: GOERTEK Inc. |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221121 Address after: 266104 No. 500, Songling Road, Laoshan District, Qingdao, Shandong Applicant after: GOERTEK TECHNOLOGY Co.,Ltd. Address before: 261031 east of Dongming Road, north of Yuqing East Street, high tech Zone, Weifang City, Shandong Province (Room 502, Geer electronics office building) Applicant before: GoerTek Optical Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |