CN109997174B - Wearable Spectral Inspection System - Google Patents

Abstract

In some embodiments, a system includes: a head-mounted frame detachably coupled to a head of a user; one or more light sources coupled to the head-mounted frame and configured to emit light of at least two different wavelengths toward the target object in a radiation field of view of the light sources; one or more electromagnetic radiation detectors coupled to the head-mounted component and configured to receive light reflected after encountering the target object; and a controller operatively coupled to the one or more light sources and the detector and configured to determine and display an output indicative of an identity or characteristic of the target object, the identity or characteristic being determined in relation to a characteristic of light emitted by the light sources measured by the detector.

Description

Wearable Spectral Inspection System

priority statement

This application claims priority to US Provisional Application No. 62/398,454, filed September 22, 2016, which is incorporated herein by reference.

incorporation by reference

This application incorporates by reference the entire contents of each of the following U.S. patent applications: U.S. Patent Application No. 15/072,341; U.S. Patent Application No. 14/690,401; U.S. Patent Application No. 14/555,858; U.S. Patent Application No. .14/555,585; U.S. Patent Application No. 13/663,466; U.S. Patent Application No. 13/684,489; U.S. Patent Application No. 14/205,126; U.S. Patent Application No. 14/641,376; US Provisional Patent Application No. 62/298,993 (US Patent Application No. 15/425,837); and US Patent Application No. 15/425,837.

technical field

The present disclosure relates to systems and methods for augmented reality using wearable components, and more particularly to configurations of augmented reality systems for identifying materials through reflected light properties.

Background technique

Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images, or parts thereof, are presented to users in such a way that they appear or can be perceived as real. Virtual reality or "VR" scenarios generally involve the presentation of digital or virtual image information opaque to otherwise actual real-world visual input; and augmented reality or "AR" scenarios generally involve the presentation of digital or virtual image information as a reflection of the actual world around the user Visual enhancements while still allowing users to essentially perceive and see the real world.

Referring to FIG. 1 , an augmented reality scene ( 4 ) is depicted in which a user of AR technology sees a real world park-like setting ( 6 ) featuring people, trees, buildings in the background, and a concrete platform ( 1120 ). In addition to these items, the user of AR technology also perceives that he "sees" a robot statue (1110) standing on a real-world platform (1120) and a flying cartoon-like avatar character (2) that looks like is the embodiment of the bumblebee, even though these elements (2, 1110) do not exist in the real world. The results show that the human visual perception system is very complex, and it is challenging to generate comfortable, natural, rich presentation of virtual image elements that facilitate together with other virtual or real-world image elements. For example, a head-mounted AR display (or head-mounted display or smart glasses) is typically at least loosely coupled to a user's head, and thus moves when the user's head moves. If the display system detects movement of the user's head, the data being displayed can be updated to account for the change in head pose. Certain aspects of a suitable AR system are disclosed, for example, in U.S. Patent Application Serial No. 14/205,126, entitled "System and method for augmented and virtual reality," which The entire contents are incorporated herein by reference, and are disclosed in the following additional disclosure relating to augmented and virtual reality systems such as those developed by Qiyue Inc. of Fort Lauderdale, Florida: Sequence U.S. Patent Application Serial No. 14/641,376; U.S. Patent Application Serial No. 14/555,585; U.S. Patent Application Serial No. 14/212,961; U.S. Patent Application Serial No. 14/690,401; US Patent Application; US Patent Application Serial No. 13/684,489, and US Patent Application Serial No. 62/298,993, each of which is hereby incorporated by reference in its entirety.

The systems and methods disclosed herein address various challenges and developments associated with AR and VR technologies.

Contents of the invention

The mixed reality system is configured to perform spectroscopy. Mixed reality (or "MR" for short) generally involves virtual objects that are integrated into and respond to the natural world. For example, in MR scenarios, AR content is occluded by real-world objects and/or is perceived as interacting with other objects (virtual or real) in the real world. Throughout this disclosure, references to AR, VR, or MR do not limit the invention, and these techniques may be applied in any context.

Some embodiments relate to a method for identifying substances (e.g., tissue, cells or properties within cells/tissues) for wearable systems. Although this disclosure primarily refers to tissue or tissue properties as the subject of analysis according to various embodiments, technologies and techniques and components are not so limited. Some embodiments utilize one or more light sources, such as electromagnetic radiation emitters coupled to a head mounted component, to emit light at one or more wavelengths in a user-selected direction. These embodiments allow continuous and even passive measurements. For example, a user wearing a head-mounted system can perform a given activity, but inward-facing sensors can detect the properties of the eyes without interfering with the activity.

For example, a user may wear a system configured to look inside the user's eye and identify or measure tissue properties of the eye, such as blood concentration in blood vessels of the eye. In other examples of inward systems, fluids such as intraocular fluid may be analyzed rather than just tissue properties. In other examples, the system may include sensors that look out towards the outside world and identify or measure tissue or material properties other than the eyes, such as the user's extremity or objects in the surrounding environment that are remote from the user.

In an outward-looking system, an eye-tracking camera coupled to the head-mounted component can determine the directional gaze the user is looking at, and the processor or controller can determine the directional gaze that the user is looking at, and the processor or controller can communicate with a real-world capture system coupled to the head-mounted component, such as a camera or depth The image captured by the sensor) correlates that gaze with the observation of a real-world target object. A light source coupled to the head-mounted system emits light away from the user, such as infrared light from an electromagnetic radiation emitter, and in some embodiments emits light to create substantially the same direction as the gaze direction determined by the eye-tracking camera. The radiation pattern is thus emitted on the target object.

In some embodiments, a real world capture system captures objects. For example, depth sensors such as vertical-cavity surface-emitting lasers can determine the contours of objects by collecting time-of-flight signals of impacting objects. Once an object is recognized in its outline by such a real world capture system, the object can be highlighted and available for labeling. In some embodiments, the camera system for a given field of view defines an area available for highlighting and marking. For example, a camera associated with the user's gaze may contain a 5 degree field of view, a 10 degree field of view, or a central visual field of view in suitable increments, preferably up to 30 degrees, within which the light source will emit light substantially.

In some embodiments, such a system further includes: one or more electromagnetic radiation detectors or photodetectors coupled to the head mounted component and configured to receive reflected light emitted from the light source and reflected from the target object; and a controller operatively coupled to the one or more electromagnetic radiation emitters and the one or more electromagnetic radiation detectors and configured to cause the one or more electromagnetic radiation emitters to emit light pulses while also causing the one or more An electromagnetic radiation detector detects the level of light absorption associated with emitted light pulses, which depends on the reflected light emitted by any received particular pulse.

In some embodiments, the system also includes a processor to match the wavelength of reflected light received by the detector from the target object to a particular material, tissue type or characteristic of the underlying tissue. In some embodiments, other light characteristics are determined, such as polarization changes or scattering effects with respect to emitted and detected light, but for purposes of this description wavelength characteristics are used as exemplary light characteristics. For example, in some embodiments, an inward-facing electromagnetic radiation emitter emits light in the infrared spectrum to the user's retina, receives the reflected light, and matches the wavelength of the reflected light to determine parameters such as the type of tissue or oxygen saturation in the tissue. physical properties. In some embodiments, the system includes an outwardly facing light source and emits infrared light toward a target object (eg, a limb of a user or a third person), receives the reflected light, and matches the reflected light wavelength to determine the observed material. For example, this outward-facing system can detect the presence of cancer cells in healthy cells. Because cancer cells or other abnormal cells reflect and absorb light differently than healthy cells, reflection of light at certain wavelengths can indicate the presence and amount of abnormalities.

In some embodiments, the controller receives the captured target object from the real world capture system and applies a tag to the target object indicative of the identified characteristic. In some embodiments, the label is a text label or prompt within the display of the head mounted unit. In some embodiments, the label is an audio prompt to the user. In some embodiments, the label is a virtual image of similar tissue, such as that cited in a medical book, superimposed near the target object for the user to prepare for comparative analysis.

In some embodiments, the head mounted component may include eyeglass frames. The spectacle frame may be a binocular spectacle frame. The one or more radiation emitters may include light sources, such as light emitting diodes. The one or more radiation emitters may include multiple light sources configured to emit electromagnetic radiation at two or more different wavelengths. The plurality of light sources may be configured to emit electromagnetic radiation at a first wavelength of about 660 nanometers and a second wavelength of about 940 nanometers. One or more radiation emitters may be configured to sequentially emit electromagnetic radiation at two different wavelengths. One or more radiation emitters may be configured to simultaneously emit electromagnetic radiation of two predetermined wavelengths. The one or more electromagnetic radiation detectors may include devices selected from the group consisting of photodiodes, photodetectors, and digital camera sensors. One or more electromagnetic radiation detectors may be positioned and oriented to receive light reflected after encountering a target object. One or more electromagnetic radiation detectors may be positioned and oriented to receive light reflected after encountering observed tissue or material; that is, whether facing inward toward the user's eyes or outward toward the user's environment, a The or more electromagnetic radiation detectors are oriented substantially in the same direction as the one or more electromagnetic radiation emitters.

The controller can also be configured to cause the plurality of light sources to emit a cyclic pattern of a first wavelength on, then a second wavelength on, and then both wavelengths off, such that the one or more electromagnetic radiation detectors detect the first and second wavelengths separately. wavelength. The controller may be configured to cause the plurality of light emitting diodes to emit a cyclic pattern of the first wavelength on, then the second wavelength on, and then both wavelengths off in a cyclic pulse pattern approximately thirty times per second.

In some embodiments, the controller may be configured to calculate a ratio of the first wavelength light measurement to the second wavelength light measurement. In some embodiments, this ratio may be further converted to an oxygen saturation reading via a lookup table based at least in part on the Beer-Lambert law. In some embodiments, the ratio is converted to a material identifier in an external lookup table, such as stored on the headset or in an absorption database module of the headset coupled to a local or remote processing module . For example, an absorbance database module for tissue-specific absorbance ratios or wavelength reflectance may be stored in a "cloud" storage system accessible to the healthcare provider and via the remote processing module. In some embodiments, the absorption database module may store the absorption characteristics (eg wavelength ratio or wavelength reflection) of certain foods and be permanently stored on the local processing module to the head mount.

In this manner, the controller may be configured to operate one or more electromagnetic radiation emitters and one or more electromagnetic radiation detectors for use as a widely used head-mounted spectroscope. The controller is operatively coupled to an optical element that is coupled to the head mounted component and viewable by the user such that an output of the controller indicative of a wavelength characteristic indicative of a particular tissue property or material is viewable by the user through the optical element . The one or more electromagnetic radiation detectors may comprise a digital image sensor comprising a plurality of pixels, wherein the controller is configured to automatically detect a subset of pixels received in an environment encountered, e.g., tissue or within tissue The light reflected after the cells. In some embodiments, such a subset of pixels is used to generate an output representative of a target object within the field of view of the digital image sensor. For example, the output may be a display label indicating the absorption level of the tissue. In some embodiments, the comparison value is displayed as an output. For example, the output may be the percent saturation of oxygen in blood from a first analysis time and the percent saturation of oxygen in a second analysis time and the rate of change recorded between the two times. In these embodiments, diseases such as diabetic retinopathy may be detected by identifying changes in measured properties over time.

In some embodiments, the controller may be configured to automatically detect the subset of pixels based at least in part on differences in reflected light brightness between signals associated with the pixels. The controller may be configured to automatically detect the subset of pixels based at least in part on differences in reflected light absorption between signals associated with the pixels. In such embodiments, such subsets may be isolated pixels and labeled for further analysis, such as for additional radiation or mapping, or a virtual image may be overlaid on such pixels to provide and display other The visual contrast of the isolated pixels of the characteristic is used to inform the user of the different characteristics of the sub-pixels identified by the system.

In some embodiments, system data collection is not only time multiplexed for pulses and recording light pulses, but is collected passively multiple times per day. In some embodiments, a GPS or other similar mapping system is coupled to the system to correlate the user's location or time of day with certain physiological data collected. For example, a user can track physiological responses relative to certain locations or activities throughout the day.

These and many other features and advantages of the present invention will be understood upon further consideration of the following drawings and description.

Description of drawings

Figure 1 illustrates certain aspects of an augmented reality system presentation to a user.

2A-2D illustrate certain aspects of various augmented reality systems for wearable computing applications, featuring head-mounted components operably coupled to local and remote processing and data components.

Figure 3 illustrates certain aspects of an example of a connection between a wearable augmented or virtual reality system and certain remote processing and/or data storage sources.

4A-4D illustrate various aspects of pulse oximetry configurations and calibration curves related to light scatter in the oxygenation of blood.

FIG. 5 illustrates a head-mounted spectroscopic inspection system integrating AR/VR functionality, according to some embodiments.

6 illustrates various aspects of a wearable AR/VR system featuring an integrated spectral inspection module, according to some embodiments.

7A-7B are example light saturation graphs indicating selective properties of passed wavelengths.

FIG. 8 illustrates a method for identifying materials or material properties with a head-mounted spectroscopic inspection system, according to some embodiments.

Detailed ways

Some AR and VR systems include processing capabilities, such as controllers or microcontrollers, and also power supplies for powering the functions of various components, and since at least some components in a wearable computing system such as an AR or VR system are in close proximity to The fact of the user's body operating them provides an opportunity to exploit some of these system components to perform some physiological monitoring with respect to the user. For example, physiological monitoring can be performed by measuring light absorption.

In conventional light absorption measurement techniques (e.g., a pulse oximeter attachable to a human finger as shown in Figure 4A or in glucose detection), light is emitted and received in a controlled and fixed direction In a controlled and fixed receiver. Light is pulsed at different wavelengths through the surrounding tissue structure, while also being detected on the other side of the tissue structure (thus measuring light properties such as absorption and scattering). In such systems, measurements of emitted light compared to measurements of detected light can provide an output proportional to or read as an output of an estimated tissue or tissue property (e.g., pulse oximetry The estimated blood oxygen saturation level of the meter) or simply the output of the material or tissue type. As shown in Figure 4D, a calibration curve depicting the ratio of the light of interest relative to other lights can also predict properties of the underlying tissue that depend on the light incident on it.

Raman spectroscopy is another technique that measures the inelastic scattering of photons released by irradiating molecules. When illuminated, specific molecules will exhibit a specific shift in wavelength, thereby presenting a unique scattering effect that can be used to measure and quantify the molecules within the sample.

Figure 4B shows a graph of the absorption spectra of oxygenated (806) and deoxygenated (808) hemoglobin, as shown by such curves (806, 808), in the red wavelength range of the electromagnetic spectrum, for example at about 660 nm, oxygen There is a marked difference in the absorption of synthetic hemoglobin and deoxygenated hemoglobin, with an inverse difference at about 940 nm in the infrared wavelength range. Pulsed radiation at such wavelengths and detection using pulse oximeters are known to exploit this difference in absorption in the determination of oxygen saturation for a particular user.

While pulse oximeters (802) are typically configured to at least partially enclose tissue structures such as a finger (804) or an earlobe, certain tabletop systems, such as those shown in Figure 4C (812), have been proposed to Observes differences in absorption by eye vessels such as retinal vessels, but can also be configured to detect properties of other tissues.

Such a configuration (812) may be referred to as a flow oximeter or spectrometer system and may include components as shown, including a camera (816), zoom lens (822), first (818) and second (820) light emitting diodes (LEDs) and one or more beam splitters (814). While it is valuable for some users, such as high altitude hikers, athletes, or people with certain cardiovascular or respiratory issues, to be able to retrieve their blood oxygen saturation as they move and engage in activities throughout the day information, or for caregivers to be able to analyze tissue in real time for potential abnormalities, but most configurations involve somewhat inconvenient wrapping of tissue structures or are not portable or wearable, regardless of other indications of tissue status or material absorption feature or does not include the gaze the user is looking at as part of its sensor's directionality (in other words, lacks selectivity of target objects for identification and analysis by spectral inspection).

Advantageously, in some embodiments, a solution is presented herein that combines the convenience of wearable computing in the form of an AR or VR system with an imaging device to determine in real-time additional tissue identification and characteristic.

Referring to Figures 2A-2D, some general component options are shown. In the section of the Detailed Description that follows the discussion regarding Figures 2A-2D, various systems, subsystems, and components are presented that address high-quality, comfortable perception of VR and/or AR for humans accessing and creating external information sources The display system target.

As shown in FIG. 2A, an AR system user (60) is depicted wearing a head-mounted assembly (58) with a frame (64) coupled to a display system (62) positioned in front of the user's eyes. ) structure is characteristic. A speaker (66) is coupled to the frame (64) in the configuration shown and is positioned near the user's ear canal (in one embodiment, another speaker (not shown) is positioned near the user's other ear canal to provide stereophonic sound) /shapeable sound control). The display (62) is operatively coupled (68), such as by wired leads or a wireless connection, to a local processing and data module (70), which may be installed in various configurations, such as being fixed attached to the frame (64), fixedly attached to a helmet or hat (80) as shown in the embodiment of FIG. Removably attached to the user's (60) torso (82), or in a strap-coupled configuration shown in the embodiment of Figure 2D, to the user's (60) hips (84).

The local processing and data module (70) may include a processor and controller (e.g., a power efficient processor or controller) and digital memory such as flash memory, both of which may be used to assist in processing, caching, and storing data , the data includes: a) data captured from sensors that may be operatively coupled to the frame (64), such as electromagnetic emitters and detectors, image capture devices (such as cameras), microphones, inertial measurement units , accelerometer, compass, GPS unit, radio, and/or gyroscope; and/or b) data acquired and/or processed using a remote processing module (72) and/or remote data repository (74), which can be After such processing or retrieval is transmitted to the display (62). The local processing and data module (70) may be operably coupled (76, 78) to a remote processing module (72) and remote data repository (74), such as via a wired or wireless communication link, such that these remote modules (72, 74) 74) are operatively coupled to each other and available as resources for the local processing and data module (70).

In one embodiment, the remote processing module (72) may include one or more relatively powerful processors or controllers configured to analyze and process data, emitted or received light characteristics, and/or or image information. In one embodiment, the remote data repository (74) may comprise a relatively large scale digital data storage facility that may be available through the Internet or other network configuration in a "cloud" resource configuration. In one embodiment, all data is stored and all calculations are performed in local processing and data modules, allowing complete autonomous use from any remote module.

Referring now to FIG. 3, a schematic diagram illustrates the coordination between cloud computing assets (46) and local processing assets, which may exist, for example, in a head-mounted assembly (58) coupled to a user's head (120) and A local processing and data module (70) coupled to the user's belt (308; thus assembly 70 may also be referred to as a "belt pack" 70), as shown in FIG. In one embodiment, cloud (46) assets such as one or more server systems (110) such as via a wired or wireless network (wireless generally preferred for mobility, wired generally preferred for certain high bandwidth or High data capacity transmission) is operatively directly coupled (115) to (40, 42) one or both of the local computing assets coupled to the user's head (120) and belt (308) as described above, the Local assets such as processor and memory configuration. These computing assets local to the user may also be operably coupled to each other via wired and/or wireless connection configurations (44), such as the wired coupling (68) discussed below with reference to FIG. 8 .

In one embodiment, in order to maintain the low inertia and small size of the subsystem mounted to the user's head (120), the primary transmission between the user and the cloud (46) may be via the belt-mounted (308) subsystem and the cloud. A link between the head-mounted (120) subsystem, where the head-mounted (120) subsystem primarily uses a wireless connection (such as an ultra-wideband ("UWB") connection) data-tether to the belt-based (308) subsystem, as currently for example in as used in personal computing peripheral connectivity applications.

Through efficient local and remote processing coordination and appropriate display means for the user, such as the user interface or user display system (62) shown in FIG. Aspects can be transmitted or "delivered" to users and updated in an efficient manner. In other words, a map of the world may be continuously updated at a storage location that may, for example, reside partly on the user's AR system and partly in a cloud resource. Maps (also called "transferable world models") can be large databases including raster images, 3-D and 2-D points, parametric information, and other information about the real world. As more and more AR users continue to capture information about their real-world environment (e.g., via cameras, sensors, IMUs, etc.), the mapping becomes more accurate and complete.

With a configuration as described above, where there is a model of the world that can be located on cloud computing resources and distributed from there, such a world can be "deliverable" to one or more users in a relatively low bandwidth form, preferably trying to deliver live video data etc. In some embodiments, the enhanced experience of a person standing near a statue (i.e., as shown in FIG. 1 ) may be informed by a cloud-based world model, a subset of which may be passed down to a The world models of the clouds and their local displays complete the view. A person sitting at a remote display device (which could be as simple as a personal computer sitting on a desk) can effectively download the same portion of information from the cloud and present it on their display. In fact, a person physically present in a park near a statue might take a remotely located friend for a walk in that park, where the friend joins in via virtual and augmented reality. The system will need to know where the street is, where the tree is, where the statue is - but about that information on the cloud, the friends who join can download it from the cloud side of the scene and then act as a local augmented reality relative to the person actually in the park Start walking together.

Three-dimensional (3-D) points can be captured from the environment, and the pose (i.e., vector and/or origin position information relative to the world) of the camera that captured those images or points can be determined so that the points or images can be identified by the pose information "Mark" or be associated with this gesture information. The pose of the second camera can then be determined using the points captured by the second camera. In other words, the second camera may be oriented and/or positioned based on a comparison with the marker image from the first camera. This knowledge can then be used to extract textures, make maps, and create a virtual copy of the real world (since then there are two cameras registered around there).

So at a basic level, in some embodiments, a human-worn system can be utilized to both capture 3-D points and generate 2-D images of those points, and these points and images can be transmitted to cloud storage and processing resources . They can also be cached locally with embedded pose information (e.g., cached labeled images); thus, the cloud can have ready (e.g., in cache available) labeled 2-D images (e.g., marked by 3-D poses) and 3-D points. If the user is looking at something dynamic (e.g., a scene with moving objects or features), he/she can also transmit additional information to the motion-related cloud (e.g., if viewing another person's face, the user can Take a texture map of the face and push it up at an optimized frequency, even though the world around you is largely static). More information on object recognizers and transferable world models, as described above, can be found in Serial No. 14 entitled "System and method for augmented and virtual reality" /205,126, the entire contents of which are incorporated herein by reference, as well as in the following additional disclosures, which relate to enhanced and Virtual Reality Systems: U.S. Patent Application Serial No. 14/641,376; U.S. Patent Application Serial No. 14/555,585; U.S. Patent Application Serial No. 14/212,961; U.S. Patent Application Serial No. 14/690,401; US Patent Application Serial No. 13/663,466; US Patent Application Serial No. 13/684,489; and US Patent Application Serial No. 62/298,993, each of which is hereby incorporated by reference in its entirety.

In some embodiments, the use of such transferable world information may allow identification and labeling of objects by spectral inspection and then transfer between users. For example, in a clinical setting, a first caregiver operating a device implementing the features of the present disclosure may map and detect cancerous tissue on a patient, and assign and apply a virtual tag very similar to a meta tag to the organize. A second caregiver, similarly wearing such a device, can then view the same population of cancerous tissue cells and receive notification of a virtual label identifying those cells without being involved in emitting light, receiving light, matching absorption features to tissue, and independently To mark one or more of the organizations.

GPS and other positioning information can be used as input to this process. It should be appreciated that highly precise positioning of the user's head, totem, gestures, haptic devices, etc. may be advantageous in displaying appropriate virtual content to the user Displaying deliverable virtual or augmented content among users in the deliverable world.

Referring to FIG. 5 , a top orthogonal view of a head-mountable assembly ( 58 ) of a wearable computing arrangement is shown featuring various integrated components for an exemplary spectroscopic inspection system. The configuration features the following components: two display elements (62 - binocular - one for each eye); two forward-oriented cameras (124) for observing and detecting the world around the user, each camera ( 124) having associated fields of view (18, 22); at least one spectroscopic inspection array (126, described in more detail in FIG. 6) having field of view (20); and a forward a directional, relatively high-resolution picture camera (156); one or more inertial measurement units (102); and a depth sensor (154) with an associated field of view (24), e.g., as described in the previously cited reference disclosure . Towards the user's eyes (12, 13) and coupled to the frame of the head mounted assembly (58) are eye tracking cameras (828, 830) and inward transmitters and receivers (832, 834). Those skilled in the art will understand that the inward emitters and receivers (832, 834) emit radiation patterns (824, 826) in the same manner as the spectroscopic inspection array (126) affects outward objects in its field of view (20). ) emits and receives light directed at the eye. These components, or a combination of all components, are operably coupled, eg, by leads, to a controller (844), which is operatively coupled (848) to a power source (846), eg, a battery.

In some embodiments, the display element (62) includes one or more waveguides (eg, a waveguide stack) that are optically transmissive and allow a user to "see" the world by receiving light from the world. The waveguide also receives light containing display information and propagates the light and directs the light to the user's eyes (12, 13), thereby displaying an image to the user. Preferably, the light propagating out of the waveguide provides a specific and defined level of wavefront divergence corresponding to the different depth planes (e.g. light forming an image of an object at a specific distance from the user has a wavefront divergence that The wavefront divergence corresponds to or substantially matches the wavefront divergence of the light that would, in reality, reach the user from the object). For example, a waveguide may have optical power and may be configured to output light with a selectively variable level of wavefront divergence. It will be appreciated that this wavefront divergence provides a cue regarding the accommodation of the eyes (12, 13). In addition, the display element (62) utilizes binocular parallax to provide further depth cues, eg, cues about the vergence of the eyes (12, 13). Advantageously, the cues on adaptation and the cues on vergence may be matched, for example, so that they both correspond to objects at the same distance from the user. This fit-vergence fit facilitates long-term wearability of the system utilizing the head-mounted component (58).

Continuing with reference to FIG. 5 , preferably each emitter ( 126 , 832 , 834 ) is configured to controllably emit two or more wavelengths of electromagnetic radiation, such as about 660 nm and about 940 nm, such as by an LED, and preferably radiates Fields (824, 826) are oriented to illuminate a target object or surface. In some embodiments, the target object is facing inward, such as the eye (12, 13), and the radiation pattern (824, 826) can be fixed or widened/narrowed to be directed toward the eye in response to eye tracking camera data points. specific area. In some embodiments, the target object is outward (e.g., away from the user) and the radiation pattern within the field of view (20) of the spectrometer array (126) conforms to the eye (12) determined from the eye-tracking cameras (828, 830). , 13) the gaze.

In some embodiments, the gaze can be understood as a vector extending from the user's eyes, such as through the lens of the eye from the fovea, the emitters (832, 834) can output infrared light with respect to the user's eyes, and the gaze from the eyes can be monitored. Reflections (eg, corneal reflections). The vector between the pupil center of the eye (eg, the display system can determine the centroid of the pupil, eg, by infrared imaging) and the reflection from the eye can be used to determine the gaze of the eye. In some embodiments, when estimating the position of the eye, since the eye has a sclera and an eyeball, the geometry can be represented as two circles layered on top of each other. An eye pointing vector can be determined or calculated based on this information. The center of rotation of the eye can also be estimated because the cross-section of the eye is circular and the sclera wobbles at a certain angle. Due to the autocorrelation of the received signal with the known transmitted signal, this can result in vector distances rather than just ray traces. The output can be viewed as a Purkinje image 1400, which can in turn be used to track eye movement.

Those skilled in the art will appreciate other ways of determining the radiation pattern within the field of view (20), such as through head posture information determined by one or more of the IMUs (102).

In some embodiments, the transmitter may be configured to emit wavelengths simultaneously or sequentially using a controlled pulse (pulsatile) emission cycle. The one or more detectors (126, 828, 830) may include photodiodes, photodetectors, and/or digital camera sensors, and are preferably positioned and oriented to receive data that has otherwise encountered the target tissue or material or object. radiation. The one or more electromagnetic radiation detectors (126, 828, 830) may comprise a digital image sensor comprising a plurality of pixels, wherein the controller (844) is configured to automatically detect a subset of pixels that receive light reflected after encountering a target object, and the controller (844) is configured to generate an output using such a subset of pixels.

In some embodiments, the output is dependent on matching received light relative to emitted light to targets from an absorption database of materials and material properties. For example, in some embodiments, the absorption database includes a plurality of absorption graphs as shown in Figures 7A and 7B. It should be understood that a database including a diagram may include an electronic representation or transformation of the information in the diagram, and that use of the term diagram herein includes such representation or transformation. Figures 7A and 7B are used as examples only, but demonstrate the various tissue properties that can be detected from a given system that emits light from a particular light source and receives light of a particular wavelength and/or light characteristic to determine what the target of observation is A particular tissue or probability of having a particular characteristic within an organization. Other graphs such as saturation curves or calibration curves can be selectively accessed by the user. For example, a user can select an absorption database for a specific light source or wavelength pattern, and then look around until the spectroscopy system identifies a material that matches the desired properties. Such an embodiment may be referred to as a "closed search" or as one that looks for a specific characteristic and then searches a database for a match to the detected light characteristic, as opposed to an "open search" that looks at any object.

The controller (844) may be configured to automatically detect a subset of pixels within the field of view (124 or 126 or 824, 826, Fig. 5) based at least in part on differences in reflected light characteristics between signals associated with the pixels. For example, the controller (844) may be configured to automatically detect a subset of pixels based at least in part on differences in reflected light absorption between signals associated with the pixels. Without being bound by theory, light striking an object will reflect, transmit (absorb) or scatter upon hitting the object such that R+T+S=1 (R=reflection from object, T=transmission to object /absorption, S=scattering from the object). If a particular subset of pixels reflects a higher proportion of light relative to surrounding sub-pixels, the controller may isolate those sub-pixels or record or register the pixel locations for these different characteristics in a memory system. In some embodiments, the pixel locations are stored as dense or sparse map points in a transferable world mapping system, such as an additional user access map of a head-mounted display system, and the subset of pixels is passed to the additional user and read by the second user. Display access and/or display on the second user display.

Referring to FIG. 6, the spectral inspection array (126) may include a light source (612) that emits light (613) toward a target object (620). In some embodiments, the light source (612) is an electromagnetic emitter such as a light emitting diode. In some embodiments, the direction of emitted light (613) is substantially the same as the gaze orientation of the user (60) or the head gesture orientation of the user (60). In some embodiments, a photodetector (614) captures reflected light (615) from a target object. In some embodiments, the processor (610), which may be the controller (844) depicted in FIG. 5, determines the absorption characteristics between the emitted light (613) and the reflected light (615) and matches characteristic. In some embodiments, the ingestion database (630) is stored on a local processing module such as the module (70) shown in FIG. 2A; in some embodiments, the ingestion database (630) is stored on a On the teleprocessing module (72) of the teleprocessing module.

For simplicity, the object (620) is depicted as an apple in FIG. 6, and while food properties have their own light absorption properties, and embodiments of the present invention can be used to identify food by their light properties, it is also contemplated that more complex uses. In some embodiments, outward facing spectroscopic inspection array (126) identifies a tissue source (624), eg, an arm depicted for illustration purposes. Emitted light (613) may strike a tissue source (624), and reflected light (615) may indicate the presence of irregular cells (626) among regular cells (625). When the light source (612) illuminates the tissue source (624), the irregular cells (626) return a different light characteristic to the light detector (614) than the regular cells (625). Irregular cells (626) may be cancerous, part of scar tissue, or even healthy cells in tissue that simply indicate or have differences from surrounding cells, such as indicative of Where a blood vessel or bone may be located. In some embodiments, regular cells constitute the majority of the cells in the sample in the assay, irregular cells constitute a minority of the sample, and the irregular cells exhibit different detectable properties than the regular cells. In some embodiments, a real-world camera capturing pixel-level images can mark such irregular cells (626). As previously mentioned, one such marking system may be a marking system applying a text image close to the irregular cells (626), and another such marking system may be a color overlay on the irregular cells (626), such as via the display element 62 (Figure 5) see.

Thus, referring again to FIG. 5 , it is proposed that a system for determining tissue properties or materials by a wearable computing system, such as a system for AR or VR, comprises: a head-mounted part (58) detachably coupled to The user's head; one or more electromagnetic radiation emitters (126, 832, 834) coupled to the head-mounted component (58) and configured to emit radiation having at least two different wavelengths in an inward or outward direction one or more electromagnetic radiation detectors (126, 828, 830) coupled to the head mounted component and configured to receive light reflected after encountering a target object; and a controller (844) that may operatively coupled to one or more electromagnetic radiation emitters (126, 832, 834) and one or more electromagnetic radiation detectors (126, 828, 830) and configured to cause the one or more electromagnetic radiation emitters to emit light pulse while also causing one or more electromagnetic radiation detectors to detect the level of light absorption associated with the emitted light pulse and produce a displayable output.

The head-mounted component (58) may include a frame configured to rest on the user's head, such as an eyeglass frame. The eyeglass frame may be a binocular eyeglass frame; an alternative embodiment may be a monocular eyeglass frame. The one or more emitters (126, 832, 834) may include a light source that emits light at multiple wavelengths, such as at least one light emitting diode or other emitter of electromagnetic radiation. The plurality of light sources may be configured to emit light at preferably two wavelengths, eg, a first wavelength of about 660 nanometers and a second wavelength of about 940 nanometers.

In some embodiments, one or more emitters (126, 832, 834) may be configured to sequentially emit light of corresponding wavelengths. In some embodiments, one or more emitters (126, 832, 834) may be configured to simultaneously emit corresponding wavelengths of light. The one or more electromagnetic radiation detectors (126, 828, 830) may include devices selected from the group consisting of photodiodes, photodetectors, and digital camera sensors. The controller (844) can also be configured to cause the plurality of light emitting diodes to emit a cyclic pattern of a first wavelength on, then a second wavelength on, then both wavelengths off, such that the one or more electromagnetic radiation detectors separately detect the first wavelength A wavelength and a second wavelength. The controller (844) can be configured to cause the plurality of light emitting diodes to emit a cyclic pattern of a first wavelength on, then a second wavelength on, then both wavelengths off in a cyclic pulse pattern approximately thirty times per second. The controller (844) may be configured to calculate a ratio of the first wavelength light measurement to the second wavelength light measurement, wherein the ratio is converted to an oxygen saturation reading via a lookup table based at least in part on the Beer-Lambert law.

The controller (844) may be configured to operate one or more emitters (126, 832, 834) and one or more electromagnetic radiation detectors (126, 828, 830) for use as a head-mounted spectrometer. The controller (844) is operatively coupled to the optical element (62), which is coupled to the head-mounted component (58) and viewable by the user such that the sensor of the controller (844) is indicative of a particular material property or tissue property. The output of can be viewed by the user through the optical element (62).

FIG. 7A is an exemplary light characteristic absorption chart that may be referenced by the absorption database (630, FIG. 6). As shown, various light source types in the visible spectrum such as IR, NIR or light emitting diodes may be optimal for detecting certain tissues and properties within the tissue. In some embodiments, the absorbance ratio or scatter in the calibration curve is calculated from emitted light to reflected light and applied to a given absorbance database (630) such as that shown in FIG. 7A to determine the underlying tissue and/or Characteristic or determine exception.

Figure 7B depicts the potential "overlap" of wavelengths. As shown, "oxygenated blood" can overlap with "deoxygenated blood" at certain wavelengths, leading to mute the results that the spectroscopic process can provide. To avoid this potential overlap, in some embodiments, a second, different wavelength of light is emitted to provide a second light source for measurement and comparison.

FIG. 8 illustrates a method ( 850 ) for using a wearable AR/VR system featuring a spectral inspection component to identify tissue or properties within tissue. The method (850) begins at (851), where the system directs a light source to a target object. In some embodiments, the orientation is such that the light source is directed inwardly at the user's eye, and this orientation may be fixed or scanned across the eye, eg, across the retina. In some embodiments, orienting is by determining the user's eye gaze or head pose and orienting the light source in substantially the same direction toward a target object within such gaze or pose field of view, or toward a characteristic landmark or target object.

In some embodiments, at (852), the light source emits light in a radiation pattern towards the target object or surface. In some embodiments, the light is pulsed at timed intervals by a timer. In some embodiments, the light source emits light of at least one wavelength, and at (854), a radiation detector, such as a photodetector, receives the reflected light. In some embodiments, the detector is also operatively coupled to a timer to indicate whether the received light was initially pulsed at a particular time to determine a change in light properties when reflected on the target object. In some embodiments, (852) begins at the same time as the mapping at (853), but this sequence does not have to be.

In some embodiments, at (853), the real world capture system may begin mapping the target object. In some embodiments, such mapping may include receiving transitive world data for the target object. In some embodiments, the mapping may include depth sensor analysis of the contour of the target object. In some embodiments, mapping may include building mesh models of items within the field of view and referencing them for potential marking. In some embodiments, the target object is not a specific object within the field of view that can be captured by the depth sensor, but a depth plane within the field of view itself.

In some embodiments, at (855), the controller analyzes the emitted light compared to the received light, eg, under the Beer-Lambert law or the optical density relationship (described below) or the scattering pattern of a calibration curve. In some embodiments, at (856), the compared optical properties are referenced in an absorption database or stored locally on the system or accessed remotely by the system to identify tissue or tissue properties of the target object. In some embodiments, the absorption database may include a saturated light graph such as the graph shown in FIG. 4B or may include a calibration curve for a particular wavelength of light.

In some embodiments, at (854), the radiation detector does not receive light of a different wavelength than the wavelength of light emitted at (852), and the controller cannot perform spectral analysis. This would happen as in Figure 7B, where the wavelengths overlap in some range for oxygenated and deoxygenated blood. In some embodiments, at (854a), no wavelength difference is detected between the emitted light and the received light, and substep (854b) is performed by emitting a different wavelength than that emitted at (852) light to start. The information of new light emitted and light received is then communicated to the controller at (855).

In some embodiments, after mapping the target object (853) and possibly concurrently with each of (852 to 856), at (857) the real world camera may additionally identify irregularities within the field of view sub-pixel. For example, in some embodiments, color contrast between pixels is detected during real-world capture at (853) and, at (857), these pixels are further altered to highlight such contrast as potentially unhealthy cells . In some embodiments, real world capture (853) detects irregular lines between clusters of pixels, and at (857), pixels bounded by the irregular lines are marked (eg, by a virtual color overlay) on the user display.

In some embodiments, the method (850) terminates (858) with the system displaying the tissue or material properties of the tissue to the user. In some embodiments, the display may include a text label virtually displayed near the target object, an audio label (630) describing the target object determined from the absorption database, or a similar tissue or tissue identified by the absorption database juxtaposed near the target object. A virtual image of the subject (630).

In some embodiments, a significant amount of spectral inspection activity is accomplished using software operated by the controller (844), such that digital image processing (e.g., by color, grayscale, and/or intensity threshold analysis using various filters) is performed. An initial task of locating a desired target (eg, blood vessel, muscle tissue, bone tissue, or other tissue and at a desired depth). Such targeting can be performed using pattern, shape recognition or texture recognition. Cancer cells or other irregular cells often have irregular borders. A camera system can identify a series of pixels within a camera's field of view (e.g., camera 124 and fields of view 18, 22 in FIG. Unhealthy cell borders. Optionally, the software and controller may be configured to use the central intensity of the target object and the intensity of surrounding objects/tissue to determine the contrast/optical density to the target object to determine abnormalities. These measurements may only be used to identify regions of interest for spectrometer scans consistent with the present disclosure, and are not necessarily a means of identifying the tissue itself. Additionally, as previously described with reference to irregular cells (626) in FIG. 6, the augmented reality system may overlay labels or color patterns within the boundaries of potentially unhealthy cells to mark/highlight them relative to surrounding healthy cells.

In some embodiments, the controller (844) may be used to calculate density ratios (contrast) and to calculate oxygen saturation from the density ratios of various pulse oximetry properties in blood vessels. Vessel optical density ("O.D.") at each of two or more emission wavelengths can be calculated using the following formula:

ODvessel = -log ₁₀ (Iv/It)

Where ODvessel is the optical density of the vessel; Iv is the intensity of the vessel; It is the intensity of the surrounding tissue.

Oxygen saturation (also called "SO2") in blood vessels can be calculated as a linear ratio of blood vessel optical densities at two wavelengths (OD ratio or "ODR") such that:

SO ₂ =ODR=OD _{first wavelength} /OD _{second wavelength}

In one embodiment, wavelengths of about 570nm (sensitive to deoxyhemoglobin) and about 600nm (sensitive to oxyhemoglobin) can be used in vascular oximetry such that SO2 = ODR = OD _600nm / OD570nm; such a formula does not take into account Adjust the ratio by the calibration factor.

The above formulas are just examples of references for calculating material properties. Those skilled in the art will understand many other organizational properties and relationships that the controller can determine.

It should be appreciated that utilizing the controller (844) to perform calculations and/or make determinations may involve performing calculations (844) locally on a processor within the controller. In some other embodiments, using the controller to perform calculations and/or make determinations (844) may involve utilizing the controller to interface with an external computing source (e.g., a source in the cloud (46) such as server (110) .

computer vision

As noted above, spectroscopic systems may be configured to detect objects or characteristics (eg, characteristics) of objects in the user's surroundings. In some embodiments, computer vision techniques may be used to detect objects or characteristics of objects present in the environment. For example, as disclosed herein, a front-facing camera of a spectroscopic inspection system may be configured to image an object, and the system may be configured to perform image analysis on the image to determine the presence of features on the object. The system may analyze images acquired by the outward-facing imaging system, absorption determinations, and/or reflected and/or scattered light measurements to perform object recognition, object pose estimation, learning, indexing, motion estimation, or image restoration, among others. One or more computer vision algorithms may be suitably selected and used to perform these tasks. Non-limiting examples of computer vision algorithms include: scale invariant feature transform (SIFT), accelerated robust feature (SURF), orientation (orient) FAST and rotate (rotate) BRIEF (ORB), binary robust Invariant Scalable Keypoint (BRISK), Fast Retinal Keypoint (FREAK), Viola-Jones Algorithm, Eigenfaces Method, Lucas-Kanade Algorithm, Horn-Schunk Algorithm, Mean-shift Algorithm, Visual Synchronous Localization and Mapping (vSLAM) Technology , sequential Bayesian estimators (e.g., Kalman filter, extended Kalman filter, etc.), bundle adjustment, self-adjusting thresholding (and other thresholding techniques), iterative closest point (ICP), semi-global matching (SGM), semi-global block matching (SGBM), histogram of feature points, various machine learning algorithms (such as support vector machine, k-nearest neighbor algorithm, naive Bayesian, neural network (including convolutional or deep neural network) ), or other supervised/unsupervised models, etc.) and so on.

As discussed herein, an object or characteristics (including properties) of an object may be detected based on one or more criteria (eg, absorbance at one or more wavelengths, light reflectance, and/or light scatter). When the spectral inspection system uses computer vision algorithms or uses data received from one or more sensor components (which may or may not be part of the spectral inspection system) to detect the presence of standards in the surrounding environment, the spectral inspection system may then signal The existence of an object or feature.

One or more of these computer vision techniques may also be used with data acquired from other environmental sensors (such as eg microphones, GPS sensors) to detect and determine various characteristics of objects detected by the sensors.

machine learning

Various machine learning algorithms can be used to learn to recognize the presence of objects or characteristics of objects. Once trained, the machine learning algorithm can be stored by the spectral inspection system. Some examples of machine learning algorithms may include supervised or unsupervised machine learning algorithms, including regression algorithms (e.g., ordinary least squares regression), instance-based algorithms (e.g., learning vector quantization), decision tree algorithms (e.g., classification and regression trees), Bayesian algorithms (e.g., Naive Bayes), clustering algorithms (e.g., k-means clustering), association rule learning algorithms (e.g., a-priori algorithms), artificial neural network algorithms (e.g., perceptrons), deep learning algorithms (e.g., deep Boltzmann machines or deep neural networks), dimensionality reduction algorithms (e.g., principal component analysis), ensemble algorithms (e.g., stacked generalization), and/or other machine learning algorithm. In some embodiments, individual models can be customized for individual data sets. For example, a wearable device may generate or store base models. The base model can be used as a starting point to generate data types specific (e.g., a particular user), data sets (e.g., sets of absorbance, light reflectance, and/or light scatter values obtained at one or more wavelengths), conditional situations, or Additional models for other variants. In some embodiments, a spectral inspection system may be configured to utilize a variety of techniques to generate models for analyzing aggregated data. Other techniques may include using predefined thresholds or data values.

Criteria for detecting objects or features of objects may include one or more threshold conditions. If analysis of data acquired by a sensor (eg, a camera or photodetector) indicates that a threshold condition is passed, the spectral inspection system may provide a signal indicative of the detected presence of an object in the surrounding environment. Threshold conditions may involve quantitative and/or qualitative measurements. For example, a threshold condition may include a score or percentage associated with the likelihood that an object and/or feature is present. The spectral inspection system can compare the score calculated from the sensor's data to a threshold score. If the score is above a threshold level, the light field system may signal the detection of an object or the presence of an object feature. In other embodiments, the spectral inspection system may signal the absence of an object or feature if the score is below a threshold.

It should be understood that each of the processes, methods, and algorithms described herein and/or depicted in the accompanying drawings can be embodied in and automated in whole or in part by a code module that is composed of one or more physical computing A system, hardware computer processor, execution of special purpose circuits; and/or electronic hardware configured to execute specific and specific computer instructions. Code modules can be compiled and linked into an executable program, installed in a dynamic link library, or written in an interpreted programming language. In some embodiments, particular operations and methods may be performed by circuitry specific to a given function. In some embodiments, the code modules may be executed by controller (844) (FIG. 5) and/or hardware in the cloud (46) (eg, server (110)).

Furthermore, certain embodiments of the disclosed functions are sufficiently mathematically, computationally, or technically complex that in order to perform the described functions (eg, due to the amount of computation or complexity involved) or to provide results in substantially real-time, Dedicated hardware or one or more physical computing devices (with appropriate proprietary executable instructions) may be necessary. For example, video may include multiple frames, each frame having millions of pixels, and specially programmed computer hardware is necessary in order to process the video data to provide the desired image processing task or application in a commercially reasonable amount of time.

Modules of code or any type of data may be stored on any type of non-transitory computer readable medium, such as physical computer memory, including hard drives, solid state memory, random access memory (RAM), read only memory (ROM), Optical discs, volatile or non-volatile memory, and combinations of the same and/or similar elements. In some embodiments, the non-transitory computer readable medium may be one of a local processing and data module (70, FIG. 2C ), a remote processing module (72, FIG. 2D ), and a remote data repository (74, FIG. 2D ). or part of multiple. Methods and modules (or data) may also be transmitted as a generated data signal (e.g., as part of a carrier wave or other analog or digital broadcast signal) over a variety of computer-readable transmission media, including wireless-based media and Wire/cable based medium and can take various forms (eg, as part of a single or multiplexed analog signal, or as multiple discrete digital data packets or frames). The results of disclosed procedures or processing steps may be stored persistently or otherwise in any type of non-transitory tangible computer memory or transmitted via a computer-readable transmission medium.

Any process, block, state, step or function in the flowcharts described herein and/or depicted in the accompanying drawings should be understood as potentially representing a code module, code segment, or code portion, which is included in the process to achieve a specific function ( One or more executable instructions such as logical or arithmetic functions) or steps. Various processes, blocks, states, steps, or functions can be combined, rearranged, added, deleted, modified, or otherwise changed according to the illustrative examples provided herein. In some embodiments, additional or different computing systems or code modules may perform some or all of the functions described herein. Nor are the methods and processes described herein limited to any particular order, and blocks, steps or states related thereto can be performed in any other order as appropriate, such as in series, in parallel, or in some other manner. Tasks or events may be added to or removed from the disclosed example embodiments. Furthermore, the separation of various system components in the embodiments described herein is for illustrative purposes and should not be construed as requiring such separation in all embodiments. It should be understood that the described program components, methods, and systems can generally be integrated together in a single computer product or packaged in multiple computer products. Many implementation variants are possible.

Various illustrative examples of the invention are described herein. Reference is made to these examples in a non-limiting sense. These examples are provided to illustrate the broader applicable aspects of the invention. Various changes may be made and equivalents may be substituted for the invention described without departing from the spirit and scope of the invention. Many modifications may be made to adapt a particular situation, material, composition of matter, process, process act or steps, or step or steps to one or more of the objectives, spirit or scope of the invention. Furthermore, as those skilled in the art will appreciate, each of the various variations described and illustrated herein has separate components and features, which can be readily combined with Any feature in several other embodiments is separated or combined. All such modifications are intended to be within the scope of the claims associated with this disclosure.

The present invention includes methods that may be performed using the subject apparatus. The method may include the act of providing such suitable means. Such provisioning may be performed by end users. In other words, the act of "providing" merely requires the end user to obtain, access, approach, locate, configure, activate, turn on, or otherwise provide the necessary means in the method. The methods described herein can be performed in any order of events described which is logically possible as well as in the order of events recited.

Exemplary aspects of the invention have been set forth above, along with details regarding material selection and fabrication. Other details of the present invention can be understood in conjunction with the above-referenced patents and publications as well as those generally known or understood by those skilled in the art. The same holds true with respect to the aspects of the basic method according to the invention in terms of additional actions which are generally or logically employed.

Additionally, although the invention has been described with reference to several examples optionally incorporating various features, the invention is not limited to the described or indicated inventions for every variation contemplated by the invention. Various changes may be made to the invention as described, and equivalents may be substituted (whether or not included herein for the sake of brevity) without departing from the spirit and scope of the invention. Further, where a range of values is provided, it is understood that each intervening value between the upper and lower limits of that range, as well as any other stated or intervening value within that stated range, is encompassed within the invention .

In addition, it is contemplated that any optional feature of variations of the invention described may be stated and claimed independently or in combination with any one or more of the features described herein. References to singular items include that there may be plural occurrences of the same item. More specifically, as used herein and in the associated claims, the singular forms "a," "an," "said," and "the" include plural referents unless expressly stated otherwise. In other words, in the above description as well as the claims associated with this disclosure, the subject matter of "at least one" of the article is allowed. It should further be noted that such claims may be drafted to exclude any optional elements. Accordingly, this statement is intended to serve as antecedent basis for the use of exclusive terms such as "solely" and "only" in conjunction with claim elements or the use of a "negative" limitation.

In the absence of such an exclusive term, the term "comprising" in claims associated with the present disclosure shall allow the inclusion of any additional elements, regardless of whether a given number of elements or additions are recited in such claims. A characteristic may be considered as changing a property of an element stated in a claim. Except as specifically defined herein, all technical and scientific terms used herein should be given the broadest commonly understood meaning possible while maintaining claim validity.

The breadth of the present invention is not limited by the examples provided and/or the subject description, but only by the scope of the language of the claims associated with this disclosure.

Claims (17)

1. A wearable spectrum inspection system comprising: a head-mounted display system detachably coupled to a user's head; at least one eye-tracking camera configured to detect the user's gaze; one or more light sources coupled to the head mounted display system and configured to emit light having at least two different wavelengths in the illuminated field of view in substantially the same direction as the detected gaze; one or more electromagnetic radiation detectors coupled to the head mounted component and configured to receive reflected light from target objects within the illuminated field of view; a controller operatively coupled to the one or more light sources and the one or more electromagnetic radiation detectors, the controller configured to cause the one or more light sources to emit pulses of light while also causing the said one or more electromagnetic radiation detectors detect light absorption levels associated with emitted light pulses and reflected light from said target object; an absorption database having light absorption properties of at least one material; and a graphics processor unit for displaying output to said user. 2. The system of claim 1, wherein the one or more light sources comprise a plurality of light emitting diodes. 3. The system of claim 1, wherein the one or more light sources are configured to emit electromagnetic radiation at two or more predetermined wavelengths. 4. The system of claim 3, wherein the one or more light sources are configured to emit electromagnetic radiation at a first wavelength of about 660 nanometers and a second wavelength of about 940 nanometers. 5. The system of claim 3, wherein the one or more light sources are configured to sequentially emit electromagnetic radiation of the two predetermined wavelengths. 6. The system of claim 3, wherein the one or more light sources are configured to simultaneously emit electromagnetic radiation of the two predetermined wavelengths. 7. The system of claim 1, wherein the controller is further configured to cause the one or more light sources to emit a first wavelength on, then a second wavelength on, then both the first and second wavelengths off A cyclic pattern such that said one or more electromagnetic radiation detectors detect said first and second wavelengths separately. 8. The system of claim 1 , wherein the controller is configured to calculate a ratio of a first wavelength light measurement to a second wavelength light measurement, and wherein the system is configured to divide the Ratios are converted to tissue properties. 9. The system of claim 8, wherein the controller is operatively coupled to an optical element coupled to the head mounted component and viewable by the user, wherein the The system is configured to provide an output based on the tissue characteristic, wherein the output is viewable by the user through the optical element. 10. The system of claim 1, wherein the one or more electromagnetic radiation detectors comprise devices selected from the group consisting of photodiodes, photodetectors. 11. The system of claim 1, wherein the one or more electromagnetic radiation detectors comprise digital image sensors. 12. The system of claim 11 , wherein the digital image sensor comprises a plurality of pixels, wherein the controller is configured to automatically detect a subset of pixels that are receiving light reflected after encountering a predetermined tissue property, and An output is generated showing the location of the subset of pixels, the output being indicative of the predetermined tissue characteristic. 13. The system of claim 1, wherein the head mounted component further comprises an inertial measurement unit positioning system. 14. The system of claim 13, wherein the inertial measurement system determines a pose orientation of the user's head. 15. The system of claim 14, wherein the illuminated field of view is at least as wide as the pose orientation. 16. The system of claim 1, wherein the head-mounted display system includes a waveguide stack configured to output light having a selectively variable level of wavefront divergence. 17. The system of claim 16, wherein the waveguide stack includes waveguides with optical power.

Images