CN118411704A - Mobile robot control method, mobile robot and storage medium - Google Patents

Abstract

The embodiment of the application discloses a mobile robot control method, which comprises the following steps: acquiring a first image of a target object; performing first spectrum transformation processing on the first image, and determining a second image corresponding to the first image; acquiring a third image of the target object; performing second spectrum transformation processing on the third image, and determining a fourth image corresponding to the third image; determining obstacle avoidance reference information according to the first image, the fourth image, the second image and the third image; and controlling the mobile robot to avoid the target object according to the obstacle avoidance reference information. According to the embodiment of the application, two groups of images with different spectrum types can be obtained through spectrum transformation according to the images with different spectrum types aiming at the same target object, and the obstacle avoidance reference information with higher accuracy and reliability is finally determined based on the two groups of images, so that the problem of accuracy deficiency existing when obstacle avoidance is carried out only by virtue of images with single spectrum types and the problem of incomplete data existing when obstacle avoidance is carried out only by virtue of two images with different spectrum types are avoided.

Description

Mobile robot control method, mobile robot and storage medium

Technical Field

The present application relates to the technical field of mobile robots, and in particular to a mobile robot control method, a mobile robot and a computer-readable storage medium.

Background technique

In the current related technologies, the obstacle avoidance methods of mobile robots in the process of performing cleaning tasks generally include binocular vision obstacle avoidance, structured light obstacle avoidance, etc. However, the advantages and disadvantages of each obstacle avoidance method are relatively obvious. For example, binocular vision obstacle avoidance is difficult to obtain high-precision depth information for target objects with no texture information or less texture information (such as mirrors, flat white walls, etc.), and structured light obstacle avoidance has problems such as missing image detail information when calculating the depth of obstacles due to the single-channel image characteristics of infrared images.

Summary of the invention

Embodiments of the present application provide a mobile robot control method, a mobile robot, and a computer-readable storage medium.

The mobile robot control method in the embodiment of the present application comprises the following steps:

Acquire a first image of the target object;

Performing a first spectral transformation process on the first image to determine a second image corresponding to the first image;

Acquiring a third image of the target object;

Performing a second spectral transformation process on the third image to determine a fourth image corresponding to the third image, wherein the first image and the third image have different spectral types, the first image and the fourth image have the same spectral type, and the second image and the third image have the same spectral type;

Determining obstacle avoidance reference information according to the first image and the fourth image, and the second image and the third image;

The mobile robot is controlled to avoid the target object according to the obstacle avoidance reference information.

In this way, the mobile robot in the implementation mode of the present application can obtain two groups of images with different spectral types through spectral transformation based on images of different spectral types for the same target object, and finally determine an obstacle avoidance reference information with higher accuracy and higher credibility based on the two groups of 4 images, thereby avoiding the lack of accuracy when only relying on images of a single spectral type for obstacle avoidance, and the problem of incomplete data when simply relying on two images of different spectral types for obstacle avoidance.

In some embodiments, the mobile robot control method further comprises:

Based on a laser device arranged on the mobile robot, a reference laser is emitted to the target object, wherein the first image and the fourth image are both formed by the reference laser reflected by the target object.

In this way, the mobile robot in the embodiment of the present application can also provide a laser light source for the target object through a laser device arranged on itself, so that the image acquired by the mobile robot can make up for the lack of texture information under a specific spectral type.

In some embodiments, the first image is an infrared light image, and the third image is a visible light image;

The performing a first spectral transformation process on the first image to determine a second image corresponding to the first image includes:

Determine a first function according to a first area on the first image and a second area on the third image corresponding to the first area, wherein the first function includes a first parameter, a second parameter, and a third parameter corresponding to three color dimensions of the visible light image;

Determine a second function according to a third area on the first image, the first function, and a fourth area on the third image corresponding to the third area, wherein the second function includes a fourth parameter, a fifth parameter, and a sixth parameter corresponding to three color dimensions of the visible light image;

The second image is determined according to the first function, the second function and the first image.

In this way, the implementation method of the present application can determine an image with the same spectral type as one of the acquired images by determining two sets of corresponding functions based on the two sets of images with different spectral types acquired by the mobile robot, thereby preparing data for determining obstacle avoidance reference information.

In some embodiments, determining the first function according to the first area on the first image and the second area on the third image corresponding to the first area includes:

The first parameter, the second parameter, and the third parameter are determined according to the pixel value of the first area and the pixel value of the second area, wherein the first area and the second area have the same image feature.

Thus, the embodiments of the present application also provide a specific method for determining the parameters in the first function.

In some embodiments, determining the second function according to the third area on the first image, the first function, and a fourth area on the third image corresponding to the third area includes:

determining a first comparison image according to the third area and the first function, wherein the first comparison image is a visible light image;

The fourth parameter, the fifth parameter and the sixth parameter are determined according to the difference between the first comparison image and the fourth region.

Thus, the embodiments of the present application also provide a specific method for determining the parameters in the second function.

In some embodiments, in some embodiments, the first image is an infrared light image, and the third image is a visible light image;

The performing a second spectral transformation process on the third image to determine a fourth image corresponding to the third image includes:

determining a third function according to a fifth area on the third image and a sixth area on the first image corresponding to the fifth area, wherein the third function includes a seventh parameter, an eighth parameter, and a ninth parameter corresponding to three color dimensions of the visible light image;

determining a fourth function according to a seventh area on the third image, the third function, and an eighth area on the first image corresponding to the seventh area, wherein the fourth function includes a tenth parameter, an eleventh parameter, and a twelfth parameter corresponding to three color dimensions of the visible light image;

The fourth image is determined according to the third function, the fourth function and the third image.

In this way, the implementation method of the present application can determine an image with the same spectral type as one of the acquired images by determining two sets of corresponding functions based on the two sets of images with different spectral types acquired by the mobile robot, thereby preparing data for determining obstacle avoidance reference information.

In some embodiments, determining the third function according to the fifth area on the third image and the sixth area on the first image corresponding to the fifth area includes:

The seventh parameter, the eighth parameter, and the ninth parameter are determined according to the pixel value of the fifth region and the pixel value of the sixth region, wherein the fifth region and the sixth region have the same image feature.

Thus, the embodiments of the present application also provide a specific method for determining the parameters in the third function.

In some embodiments, determining the fourth function according to the seventh area on the third image, the third function, and an eighth area on the first image corresponding to the seventh area includes:

determining a second comparison image according to the seventh area and the third function, wherein the second comparison image is an infrared light image;

The tenth parameter, the eleventh parameter and the twelfth parameter are determined according to the difference between the second comparison image and the eighth region.

Thus, the embodiments of the present application also provide a specific method for determining the parameters in the fourth function.

In some embodiments, determining obstacle avoidance reference information according to the first image and the fourth image, and the second image and the third image, includes:

determining first depth information according to the first image and the fourth image;

determining second depth information according to the second image and the third image;

The obstacle avoidance reference information is determined according to the first depth information and the second depth information.

In this way, the implementation method of the present application can group the acquired images and the further determined images in pairs according to the spectral type, and determine the depth information of a target object based on the two groups of images respectively, and finally determine the obstacle avoidance reference information based on the two groups of depth information to improve the accuracy of the obstacle avoidance reference information.

In some implementations, determining the obstacle avoidance reference information according to the first depth information and the second depth information includes:

determining first loss information according to the first depth information and a second function, wherein the second function is determined according to a third area on the first image, a first function, and a fourth area on the third image corresponding to the third area, and the first function is determined according to a first area on the first image and a second area on the third image corresponding to the first area;

determining second loss information according to the second depth information and a fourth function, wherein the fourth function is determined according to a seventh area on the third image, a third function, and an eighth area on the first image corresponding to the seventh area, and the third function is determined according to a fifth area on the third image and a sixth area on the first image corresponding to the fifth area;

When the difference between the first loss information and the second loss information is within a preset threshold range, determining an average value of the first loss information and the second loss information as the obstacle avoidance reference information; or

When the difference between the first loss information and the second loss information is outside the preset threshold range, the one corresponding to the larger depth information of the first loss information and the second loss information is determined as the obstacle avoidance reference information.

Thus, the embodiments of the present application also provide a method for specifically determining obstacle avoidance reference information based on two sets of depth information.

The mobile robot in the implementation mode of the present application includes a visible light camera device, an infrared light camera device and a laser device; the mobile robot also includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the method as described above is implemented.

The computer-readable storage medium in the embodiments of the present application stores a computer program, and when the computer program is executed by one or more processors, the above method is implemented.

Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

FIG1 is a flow chart of a mobile robot control method according to an embodiment of the present application;

FIG2 is a second flow chart of the mobile robot control method in the embodiment of the present application;

FIG3 is a schematic diagram of one of the application scenarios of the mobile robot control method in an embodiment of the present application;

FIG4 is a second schematic diagram of an application scenario of the mobile robot control method in an embodiment of the present application;

FIG5 is a third flow chart of the mobile robot control method in the embodiment of the present application;

FIG6 is a fourth flow chart of the mobile robot control method in the embodiment of the present application;

FIG7 is a fifth flow chart of the mobile robot control method in the embodiment of the present application;

FIG8 is a sixth flow chart of the mobile robot control method in the embodiment of the present application;

FIG9 is a third schematic diagram of an application scenario of the mobile robot control method in an embodiment of the present application;

FIG10 is a seventh flow chart of a mobile robot control method in an embodiment of the present application;

FIG. 11 is a schematic diagram of the structure of a mobile robot in an embodiment of the present application.

Detailed ways

The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

Please refer to FIG1 , the mobile robot control method in the embodiment of the present application specifically includes the following steps:

01: Get the first image of the target object;

02: Perform a first spectrum transformation process on the first image to determine a second image corresponding to the first image;

03: Acquire the third image of the target object;

04: Perform a second spectrum transformation process on the third image to determine a fourth image corresponding to the third image.

The first image and the third image have different spectral types, the first image and the fourth image have the same spectral type, and the second image and the third image have the same spectral type;

05: Determine obstacle avoidance reference information according to the first image and the fourth image, and the second image and the third image;

06: Control the mobile robot to avoid the target object based on the obstacle avoidance reference information.

The mobile robot control device in the embodiment of the present application can implement the above-mentioned mobile robot control method. Specifically, the mobile robot control device includes an image acquisition module, a spectral transformation module, an information determination module and a motion control module, wherein the image acquisition module is used to acquire a first image of the target object, and is used to acquire a third image of the target object, the spectral transformation module is used to perform a first spectral transformation process on the first image to determine a second image corresponding to the first image, and is used to perform a second spectral transformation process on the third image to determine a fourth image corresponding to the third image, the information determination module is used to determine obstacle avoidance reference information based on the first image and the fourth image, and the second image and the third image, and the motion control module is used to control the mobile robot to avoid the target object according to the obstacle avoidance reference information.

The mobile robot in the embodiment of the present application includes a visible light camera and an infrared light camera, which can be used to obtain a first image and a third image of a target object, respectively. In addition, the mobile robot also includes a memory and a processor, which can implement the above-mentioned mobile robot control method, wherein the memory stores a computer program, and the processor is used to obtain a first image of the target object, and to perform a first spectral transformation process on the first image to determine a second image corresponding to the first image, and to obtain a third image of the target object, and to perform a second spectral transformation process on the third image to determine a fourth image corresponding to the third image, and to determine obstacle avoidance reference information based on the first image and the fourth image, and the second image and the third image, and to control the mobile robot to avoid the target object according to the obstacle avoidance reference information.

Specifically, the current obstacle avoidance solutions for mobile robots include binocular vision obstacle avoidance, structured light obstacle avoidance, infrared obstacle avoidance, etc.

Binocular vision obstacle avoidance uses two left and right cameras of the same specifications set on the same horizontal line to simultaneously capture images of the same target object at the same time (such as two RGB cameras of the same specifications, or two IR cameras of the same specifications, etc.), obtain two camera patterns of the same specifications, and then obtain the depth information in the image through a stereo matching algorithm. The movement of the mobile robot is controlled according to the above depth information to achieve obstacle avoidance.

Infrared structured light obstacle avoidance works by actively emitting infrared light sources and photographing patterns with an infrared camera. Depending on the distance to the object, the reflected light source information will undergo different deformations. The depth value can be obtained based on the different pixel deformation amounts in the photographed pattern to achieve obstacle avoidance.

Infrared structured light obstacle avoidance and binocular obstacle avoidance solutions each have their own advantages. Binocular vision obstacle avoidance can obtain higher accuracy and more depth information, but binocular vision is passive reception, and binocular vision will fail in environments without texture information (such as mirrors or flat white walls).

Infrared structured light obstacle avoidance is generally achieved by active laser emission and a single infrared camera receiving images, which can cover scenes without textures. However, in the structured light obstacle avoidance method currently used by mobile robots in related technologies, the laser emission method is mostly active emission of single- or dual-line lasers, but obtaining depth information is often a function in single-line scenes. Due to the single-channel characteristics of infrared light images, the depth information obtained through infrared structured light images is not as rich as the depth information obtained by obtaining RGB images based on binocular vision.

In order to improve the reliability of the depth information determination process and the accuracy of the depth information, the present application proposes a mobile robot control method.

First, the mobile robot obtains two images (corresponding to the first image and the third image) of the same target object based on a pair of cameras of different spectral types provided on the mobile robot, and the two images each have the same spectral type as the corresponding camera. For example, in some examples, the mobile robot is based on a visible light camera and an infrared light camera provided on the mobile robot, the visible light camera obtains an RGB visible light image, and the infrared light camera obtains an IR infrared light image.

Then, based on the first image and the third image, an image with a spectrum type different from its own is determined respectively. Moreover, in order to achieve the purpose of improving the depth information of the generated target object, the generated image should be the same as another image acquired by the camera device, that is, the image generated based on the first image has the same spectrum type as the third image, and the image generated based on the third image has the same spectrum type as the first image. For example, based on the above example, an IR infrared light image corresponding to itself in image content is determined based on the acquired RGB visible light image, and then an RGB visible light image corresponding to itself in image content is determined based on the acquired IR infrared light image.

Next, the images with the same spectral type in the above four images are grouped separately, and the depth information of the target object (corresponding to the obstacle avoidance reference information) is determined by grouping calculation and then integrating the data, so as to control the mobile robot to judge the current situation between the target object and itself according to the above depth information, and then assist in controlling the mobile robot to avoid obstacles. The above scheme can not only retain the advantages of high accuracy of RGB image depth judgment, but also retain the advantages of high reliability of IR image in texture judgment. At the same time, it can also repeatedly calculate and integrate the depth information from two different spectral types, effectively improving the accuracy and reliability of obstacle avoidance related information.

It should be noted that the execution order of acquiring the first and third images can be changed according to actual conditions, or can be performed simultaneously. In addition, the implementation steps of determining the second and fourth images according to the first and third images only need to be performed after the corresponding images are acquired. The specific execution order can be changed according to actual conditions, or can be performed simultaneously. This embodiment describes only an exemplary execution order of the steps, and does not limit the execution order.

In this way, the mobile robot in the implementation mode of the present application can obtain two groups of images with different spectral types through spectral transformation based on images of different spectral types for the same target object, and finally determine an obstacle avoidance reference information with higher accuracy and higher credibility based on the two groups of 4 images, thereby avoiding the lack of accuracy when only relying on images of a single spectral type for obstacle avoidance, and the problem of incomplete data when simply relying on two images of different spectral types for obstacle avoidance.

In some embodiments, the mobile robot control method further comprises:

Based on the laser device set on the mobile robot, a reference laser is emitted to the target object.

The first image and the fourth image are both formed by the target object reflecting the reference laser.

In some embodiments, the mobile robot control device further includes a laser emission module, which is specifically configured to emit a reference laser to a target object based on a laser device disposed on the mobile robot.

In some embodiments, the mobile robot further comprises a laser device, and the laser device can emit a reference laser to the target object. In addition, the processor is further configured to emit a reference laser to the target object based on the laser device disposed on the mobile robot.

Specifically, it is common sense that when an active light source is not used and only ambient light sources are used to acquire images, traditional infrared camera binoculars and RGB camera binoculars cannot obtain effective disparity information in texture-free scenes, that is, they cannot obtain object depth information. For example, when a visible light camera device acquires an RGB image, if the target object has fewer texture features, it may be impossible to effectively obtain disparity information based on the RGB image. For example, in a scenario where the target object is a mirror or a white wall with missing textures, determining the depth information based solely on the RGB image will result in a large error due to the lack of texture information.

In addition, if the ambient light intensity is low, in order to ensure the quality of the image, it is generally necessary to fill in the target object with light. However, in some scenarios, using visible white light for fill in will cause the amount of light in the current environment to be too large, which will have an adverse effect on the user's normal life.

Taking into account the imaging quality of images of different spectral types and minimizing interference with the normal life of users, in some examples, a laser device can be set on the mobile robot, and the laser device can be used to emit a reference laser to the target object to fill in the light of the target object. Generally speaking, the spectral type of the above-mentioned reference laser is the same as the spectral type of one of the camera devices on the mobile robot. For example, illustratively, the spectral type of the reference laser is infrared light, and when the mobile robot acquires the image of the target object, the texture information can be reflected by the infrared light image formed by the infrared light camera device based on the reflection of the reference laser by the target object. In this way, it is possible to avoid the phenomenon that large errors are prone to occur when determining the depth information based on the RGB image when the texture information of the target object is insufficient, and to reduce interference with the user's life by using non-visible light fill-in.

In this way, the mobile robot in the embodiment of the present application can also provide a laser light source for the target object through a laser device arranged on itself, so that the image acquired by the mobile robot can make up for the lack of texture information under a specific spectral type.

In some embodiments, the first image is an infrared light image, and the third image is a visible light image.

Please refer to Figure 2. Based on this, step 02 includes:

021: determining a first function according to a first area on the first image and a second area on the third image corresponding to the first area,

The first function includes a first parameter, a second parameter and a third parameter corresponding to three color dimensions of the visible light image;

022: determining a second function according to the third area on the first image, the first function, and a fourth area on the third image corresponding to the third area,

wherein the second function includes a fourth parameter, a fifth parameter, and a sixth parameter corresponding to three color dimensions of the visible light image;

023: Determine the second image according to the first function, the second function and the first image.

In some embodiments, the spectral transformation module is also used to determine a first function based on a first area on the first image and a second area on the third image corresponding to the first area, and to determine a second function based on the third area on the first image, the first function, and a fourth area on the third image corresponding to the third area, and to determine a second image based on the first function, the second function, and the first image.

In some embodiments, the processor is also used to determine a first function based on a first area on the first image and a second area on the third image corresponding to the first area, and to determine a second function based on the third area on the first image, the first function, and a fourth area on the third image corresponding to the third area, and to determine a second image based on the first function, the second function, and the first image.

Specifically, the following is a specific description of the process of determining an image with a different spectral type from the acquired image. For the convenience of explanation, in the following description of the specific implementation method, it is assumed that the first image in the above implementation method is an infrared light image (IR image) and the third image is a visible light image (RGB image). The actual spectral types of the first image and the third image can be adjusted according to actual conditions, and this application does not make specific limitations, and only uses IR images and RGB images as exemplary descriptions.

Please refer to Figure 3. The mobile robot is provided with an infrared light camera (infrared camera), a visible light camera (RGB camera) and a laser device. The mobile robot uses the laser device to emit a reference laser to the target object to implement fill light. The infrared camera and the RGB camera respectively obtain an IR image and an RGB image of the target object, wherein the IR image is formed by the target object reflecting the reference laser and obtained based on the infrared camera, and the RGB image can also be obtained using a laser fill light solution. Then, based on the above-obtained IR image and RGB image as data, a function for spectral type conversion is determined, and a transformed RGB image (corresponding to the second image) after spectral type conversion is obtained according to the obtained function and IR image.

Specifically, the above-mentioned functions for spectral type conversion generally include two, one is a mapping function G1 (corresponding to the first function) for image conversion, and the other is a loss function Loss1 (corresponding to the second function) for avoiding image information loss or information redundancy. The input side data of the above two functions are the above-mentioned IR images, and the superposition of the output results of the two functions is the above-mentioned transformed RGB image. The mapping function G1 includes three parameters (corresponding to the first parameter, the second parameter and the third parameter), and each parameter corresponds to one of the three color channels of RGB. Similarly, the loss function Loss1 also includes three parameters (corresponding to the fourth parameter, the fifth parameter and the sixth parameter), and each parameter also corresponds to one of the three color channels of RGB.

In order to determine the mapping function G1 and the loss function Loss1, it is necessary to determine the six parameters from the first to the sixth mentioned above. When the six parameters are determined, the mapping function G and the loss function Loss can be determined in combination with the preset calculation method of the function, and the IR image (first image) is used as input for calculation, and the transformed RGB image can be obtained. The above-mentioned spectral transformation processing based on the mapping function G1 and the loss function Loss1 for the IR image corresponds to the first spectral transformation processing.

In the process of determining the first to sixth parameters, the camera positions on the mobile robot are different, resulting in parallax between the acquired IR image and the RGB image, so the above parameters cannot be directly determined by directly comparing the image contents at the same position on the two images. In order to determine the image area that can be directly used for image comparison, it is necessary to perform image feature analysis on the acquired IR image and RGB image to determine the corresponding regions of interest (ROI) on the two images that have the same image features.

Please refer to Figure 4, which exemplarily shows two images of the same target object acquired by two sets of camera devices set on a mobile robot in a certain environment, wherein the first image is an IR image and the third image is an RGB image. Since the two sets of camera devices are located at different positions on the mobile robot, it is necessary to perform image feature analysis on the two sets of images, and use regions with similar features as corresponding ROIs on the two images. The image feature analysis method, that is, the specific method for determining ROI, can adopt relevant technical means in the current relevant technology, and this application does not make specific limitations.

For example, in FIG4 , the R1 feature and the R1-1 feature have similar image features, so the region where R1 is located and the region where R1-1 is located are determined as corresponding ROIs, and the R2 feature and the R2-1 feature have similar image features, so the region where R2 is located and the region where R2-1 is located are determined as corresponding ROIs.

In practical applications, there are more than one group of corresponding ROIs that can be determined on the first image and the third image. Generally, any one group is selected as the data basis for calculating the first, second and third parameters. After the first, second and third parameters are determined, that is, the mapping function G1 is determined, other ROI groups and the mapping function G1 are used for calculation, and the fourth, fifth and sixth parameters and the loss function Loss1 are determined using the calculated data. In addition, in order to improve the accuracy of the loss function Loss1, other ROIs on the first image and the third image can also be selected to repeatedly calculate and verify the fourth, fifth and sixth parameters in the loss function Loss1 until the loss function Loss1 meets expectations.

For example, in FIG4 , the first, second, and third parameters are calculated with the R1 feature (corresponding to the first region) and the R1-1 feature (corresponding to the second region) as the corresponding ROI group, and the mapping function G1 is obtained. Then, with R2 and R2-1 as the corresponding ROI group, the R2 feature (corresponding to the third region) is used as the input side of the mapping function G1 for calculation, and finally the output value of the mapping function G1 is compared with the R2-1 feature (corresponding to the fourth region), so that the fourth, fifth, and sixth parameters can be obtained, and the loss function Loss1 is finally determined.

Finally, according to the above-mentioned mapping function G1 and loss function Loss1, the IR image is used as the input side data of the above-mentioned two groups of functions for calculation, and the superposition of the calculation results of the two groups of functions is the transformed RGB image (corresponding to the second image).

In this way, the present application can determine an image with the same spectral type as one of the acquired images by determining two sets of corresponding functions based on the two sets of images with different spectral types acquired by the mobile robot, thereby preparing data for determining obstacle avoidance reference information.

Referring to FIG. 5 , in some embodiments, step 021 includes:

0211: Determine a first parameter, a second parameter, and a third parameter according to the pixel value of the first area and the pixel value of the second area,

wherein the first region and the second region have the same image features;

Additionally, in some embodiments, step 022 includes:

0221: Determine a first comparison image according to the third area and the first function,

The first comparison image is a visible light image;

0222: Determine a fourth parameter, a fifth parameter, and a sixth parameter according to the difference between the first comparison image and the fourth region.

In some embodiments, the spectral transformation module is also used to determine the first parameter, the second parameter, and the third parameter based on the pixel value of the first area and the pixel value of the second area, and to determine the first comparison image based on the third area and the first function, and to determine the fourth parameter, the fifth parameter, and the sixth parameter based on the difference between the first comparison image and the fourth area.

In some embodiments, the processor is further used to determine the first parameter, the second parameter, and the third parameter based on the pixel value of the first area and the pixel value of the second area, and to determine the first comparison image based on the third area and the first function, and to determine the fourth parameter, the fifth parameter, and the sixth parameter based on the difference between the first comparison image and the fourth area.

Specifically, the following describes in detail the method for calculating the mapping function G1 and the loss function Loss1. Based on the above implementation and examples, please refer to Figure 4. First, select R1 and R1-1 as corresponding ROIs to calculate the mapping function G1. Exemplarily, the specific calculation method is: for the R1 feature and R1-1, select each pixel within the m×n range in the area where the R1 feature is located and each pixel within the corresponding m×n range in the area where the R1-1 feature is located, where m and n are the pixel values in the horizontal and vertical directions of the area where the R1 feature and the R1-1 feature are located, respectively.

Then, the interpolation calculation method in the current relevant technology is used to calculate the first, second and third parameters corresponding to the three color channels of the RGB image respectively, so as to determine the mapping function G1, wherein the first, second and third parameters correspond to which color channel of R, G and B respectively, which can be set according to actual conditions, and this application does not make specific limitations.

Next, the R2 feature on the IR image is used as the input side data of the mapping function G1 and calculated to obtain a visible light comparison image (corresponding to the first comparison image), and then a comparison analysis is performed based on the above-mentioned visible light comparison image and the R2-1 feature to determine the difference between the visible light comparison image and the R2-1 feature, and data conversion is performed based on the above-mentioned difference, and the above-mentioned fourth, fifth and sixth parameters are calculated corresponding to the three color channels of the RGB image respectively, so as to determine the loss function Loss1, wherein the fourth, fifth and sixth parameters correspond to which color channel of R, G, and B respectively, which can be set according to actual conditions, and this application does not make specific limitations.

Thus, the embodiments of the present application also provide a specific method for determining the parameters in the corresponding function.

Referring to FIG. 6 , in some embodiments, step 04 includes:

041: Determine a third function according to a fifth area on the third image and a sixth area on the first image corresponding to the fifth area,

The third function includes a seventh parameter, an eighth parameter and a ninth parameter corresponding to three color dimensions of the visible light image;

042: Determine a fourth function according to the seventh area on the third image, the third function, and the eighth area on the first image corresponding to the seventh area,

wherein the fourth function includes a tenth parameter, an eleventh parameter, and a twelfth parameter corresponding to three color dimensions of the visible light image;

043: Determine a fourth image according to the third function, the fourth function and the third image.

In some embodiments, the spectral transformation module is also used to determine a third function based on a fifth area on the third image and a sixth area on the first image corresponding to the fifth area, and to determine a fourth function based on a seventh area on the third image, a third function, and an eighth area on the first image corresponding to the seventh area, and to determine a fourth image based on the third function, the fourth function, and the third image.

In some embodiments, the processor is also used to determine a third function based on a fifth area on the third image and a sixth area on the first image corresponding to the fifth area, and to determine a fourth function based on a seventh area on the third image, a third function, and an eighth area on the first image corresponding to the seventh area, and to determine a fourth image based on the third function, the fourth function, and the third image.

Specifically, on the basis of the above-mentioned implementation, please refer to FIG3 , the mobile robot is provided with an infrared light camera device (infrared camera), a visible light camera device (RGB camera) and a laser device, the mobile robot transmits a reference laser to the target object through the laser device to implement fill light, the infrared camera and the RGB camera respectively obtain the IR image and the RGB image of the target object, wherein the IR image is formed by the target object reflecting the above-mentioned reference laser. Then, based on the above-mentioned RGB image and IR image obtained as data, a function for spectral type conversion is determined, and a converted IR image (corresponding to the fourth image) after spectral type conversion is obtained according to the obtained function and RGB image. The above-mentioned spectral conversion processing based on the mapping function G2 and the loss function Loss2 for the RGB image corresponds to the second spectral conversion processing.

Specifically, the above-mentioned functions for spectral type conversion generally include two, one is the mapping function G2 (corresponding to the third function) for image conversion, and the other is the loss function Loss2 (corresponding to the fourth function) for avoiding image information loss or information redundancy. The input side data of the above two functions are the above-mentioned RGB images, and the superposition of the output results of the two functions is the above-mentioned transformed IR image. The mapping function G2 includes three parameters (corresponding to the seventh parameter, the eighth parameter and the ninth parameter), and each parameter corresponds to one of the three color channels of RGB. Similarly, the loss function Loss2 also includes three parameters (corresponding to the tenth parameter, the eleventh parameter and the twelfth parameter), and each parameter also corresponds to one of the three color channels of RGB.

In order to determine the mapping function G2 and the loss function Loss2, it is necessary to determine the above-mentioned six parameters from the seventh to the twelfth. When the above six parameters are determined, the mapping function G2 and the loss function Loss2 can be determined in combination with the preset calculation method of the function, and the RGB image (the third image) is used as input for calculation, and the transformed IR image can be obtained.

In the process of determining the seventh to twelfth parameters, a total of six parameters, due to the different positions of the cameras set on the mobile robot, there is a parallax between the acquired IR image and the RGB image, so the above parameters cannot be directly determined by directly comparing the image content at the same position on the two images. In order to determine the image area that can be directly used for image comparison, it is necessary to perform image feature analysis on the acquired IR image and RGB image to determine the corresponding regions of interest (ROI) on the two images that have the same image features.

In practical applications, there are more than one group of corresponding ROIs that can be determined on the first image and the third image. Generally, one group is selected as the data basis for calculating the seventh, eighth and ninth parameters. After the seventh, eighth and ninth parameters are determined, that is, the mapping function G2 is determined, other ROI groups and mapping function G2 are used for calculation, and the tenth, eleventh and twelfth parameters and loss function Loss2 are determined using the calculated data. In addition, in order to improve the accuracy of the loss function Loss2, other ROIs on the first image and the third image can also be selected to repeatedly calculate and verify the tenth, eleventh and twelfth parameters in the loss function Loss2 until the loss function Loss2 meets expectations.

For example, in FIG4 , the seventh, eighth, and ninth parameters are calculated with the R1-1 feature (corresponding to the fifth region) and the R1 feature (corresponding to the sixth region) as the corresponding ROI group, and the mapping function G2 is obtained. Then, the R2-1 feature (corresponding to the seventh region) and the R2 feature (corresponding to the eighth region) are used as the corresponding ROI group, and the R2-1 feature is used as the input side of the mapping function G2 for calculation. Finally, the output value of the mapping function G2 is compared with the R2 feature, and the tenth, eleventh, and twelfth parameters can be obtained, and the loss function Loss2 is finally determined.

Finally, according to the above-mentioned mapping function G2 and loss function Loss2, the RGB image is used as the input side data of the above-mentioned two groups of functions for calculation, and the superposition of the calculation results of the two groups of functions is the transformed IR image (corresponding to the fourth image).

In this way, the present application can determine an image with the same spectral type as one of the acquired images by determining two sets of corresponding functions based on the two sets of images with different spectral types acquired by the mobile robot, thereby preparing data for determining obstacle avoidance reference information.

Referring to FIG. 7 , in some embodiments, step 041 includes:

0411: Determine the seventh parameter, the eighth parameter, and the ninth parameter according to the pixel value of the fifth area and the pixel value of the sixth area,

The fifth region and the sixth region have the same image features;

Additionally, in some embodiments, step 042 includes:

0421: Determine a second comparison image according to the seventh region and the third function,

The second comparison image is an infrared light image;

0422: Determine a tenth parameter, an eleventh parameter, and a twelfth parameter according to the difference between the second comparison image and the eighth region.

In some embodiments, the spectral transformation module is also used to determine the seventh parameter, the eighth parameter, and the ninth parameter based on the pixel value of the fifth area and the pixel value of the sixth area, and to determine the second comparison image based on the seventh area and the third function, and to determine the tenth parameter, the eleventh parameter, and the twelfth parameter based on the difference between the second comparison image and the eighth area.

In some embodiments, the processor is further used to determine a seventh parameter, an eighth parameter, and a ninth parameter based on the pixel value of the fifth region and the pixel value of the sixth region, and to determine a second comparison image based on the seventh region and the third function, and to determine a tenth parameter, an eleventh parameter, and a twelfth parameter based on the difference between the second comparison image and the eighth region.

Specifically, the following describes in detail the method for calculating the mapping function G2 and the loss function Loss2. Based on the above implementation and examples, please refer to Figure 4. First, select R1-1 and R1 as corresponding ROIs to calculate the mapping function G2. Exemplarily, the specific calculation method is: for the R1-1 feature and R1, select each pixel within the m×n range in the area where the R1-1 feature is located and each pixel within the corresponding m×n range in the area where the R1-1 feature is located, where m and n are the pixel values in the horizontal and vertical directions of the area where the R1-1 feature and the R1 feature are located, respectively.

Then, the interpolation calculation method in the current relevant technology is used to calculate the seventh, eighth and ninth parameters corresponding to the three color channels of the RGB image respectively, so as to determine the mapping function G1, wherein the seventh, eighth and ninth parameters correspond to which color channel of R, G and B respectively, which can be set according to actual conditions, and this application does not make specific limitations.

Next, the R2-1 feature on the RGB image is used as the input side data of the mapping function G2 and calculated to obtain an infrared light comparison image (corresponding to the second comparison image), and then a comparison analysis is performed based on the above infrared light comparison image and the R2 feature to determine the difference between the infrared light comparison image and the R2 feature, and data conversion is performed based on the above difference, and the tenth, eleventh and twelfth parameters are calculated corresponding to the three color channels of the RGB image respectively, so as to determine the loss function Loss2, wherein the tenth, eleventh and twelfth parameters correspond to which color channel of R, G and B respectively, which can be set according to actual conditions, and this application does not make specific limitations.

Thus, the embodiments of the present application also provide a specific method for determining the parameters in the corresponding function.

Please refer to FIG. 8 , in some embodiments, step 05 further includes:

051: Determine first depth information according to the first image and the fourth image;

052: Determine second depth information according to the second image and the third image;

053: Determine obstacle avoidance reference information according to the first depth information and the second depth information.

In some embodiments, the information determination module is further used to determine first depth information based on the first image and the fourth image, to determine second depth information based on the second image and the third image, and to determine obstacle avoidance reference information based on the first depth information and the second depth information.

In some embodiments, the processor is further used to determine first depth information based on the first image and the fourth image, to determine second depth information based on the second image and the third image, and to determine obstacle avoidance reference information based on the first depth information and the second depth information.

Specifically, on the basis of the above-mentioned implementation mode, based on the acquired IR image (corresponding to the first image) and RGB image (corresponding to the third image), the transformed RGB image (corresponding to the second image) and the transformed IR image (corresponding to the fourth image) are determined, and according to the spectral type, the images are divided into IR group and RGB group, and the depth information of the target object relative to the mobile robot is calculated based on the two groups of images.

For the calculation principle of depth information, please refer to FIG9 . FIG9 schematically shows the process of determining the depth information of the same target object by any two cameras. Exemplarily, the coordinate system shown in FIG9 takes a point on the camera device 1 as the origin, and exemplarily takes the lower left corner endpoint of the camera device 1 in FIG9 as the origin of the above coordinate system. The camera device 1 and the camera device 2 (which can be set as the RGB camera and the IR camera in the above embodiment) are set on the horizontal axis of the above coordinate system, and the distance between the target object and the horizontal axis is the depth information to be measured Depth, the distance between the straight lines where the optical centers of the camera device 1 and the camera device 2 are located is Base, and the focal lengths of the camera device 1 and the camera device 2 are both f. In the above case, there is a parallax x between the image of the target object formed by the camera device 1 and the image of the target object formed by the camera device 2. In order to obtain the parallax x, the two sets of images can be overlapped, and the parallax between the two images of the target object on the overlapped image is calculated, that is, the above parallax x can be obtained. The data forms of the depth information Depth and the disparity x may be scalar, vector or image matrix information. The above data forms may be adjusted according to actual conditions and are not specifically limited in this application.

When the above data are obtained, the depth information Depth is calculated as follows:

On the basis of the above principles and implementation methods, since the transformed RGB image (corresponding to the second image) is determined based on the IR image (corresponding to the first image), the transformed IR image (corresponding to the fourth image) is determined based on the RGB image (corresponding to the third image), and the images of the target object on the IR image and the RGB image have the above-mentioned parallax, the images of the target object on the two images with the same spectral type in the above-mentioned four images also have parallax.

Therefore, when the IR image and the transformed IR image are taken as a group, and the RGB image and the transformed RGB image are taken as a group, and the depth information Depth is calculated through the above two groups of images, the focal length f and the distance Base of the straight line where the optical center of the camera device is located are both known constants. It is only necessary to know the parallax x of the image of the target object in the group of images to determine the depth information Depth in the corresponding situation according to the above calculation method. Therefore, the depth information Depth1 (corresponding to the first depth information) under infrared light can be determined based on the IR image and the transformed IR image, and the depth information Depth2 (corresponding to the second depth information) under visible light can be determined based on the RGB image and the transformed RGB image. Finally, after data processing based on Depth1 and Depth2, the depth information Depth* (corresponding to the obstacle avoidance reference information) that can be directly used for obstacle avoidance is determined.

It should be noted that the specific execution order of the above steps of determining Depth1 and Depth2 can be changed according to actual conditions, or can be performed simultaneously. This embodiment describes only an exemplary execution order of the steps, and is not a limitation on the execution order.

In this way, the implementation method of the present application can group the acquired images and the further determined images in pairs according to the spectral type, and determine the depth information of a target object based on the two groups of images respectively, and finally determine the obstacle avoidance reference information based on the two groups of depth information to improve the accuracy of the obstacle avoidance reference information.

Please refer to FIG. 10 , in some embodiments, step 053 further includes:

0531: Determine first loss information according to the first depth information and the second function,

The second function is determined according to the third area on the first image, the first function, and a fourth area on the third image corresponding to the third area, and the first function is determined according to the first area on the first image and a second area on the third image corresponding to the first area;

0532: Determine second loss information according to the second depth information and the fourth function,

The fourth function is determined according to the seventh area on the third image, the third function, and the eighth area on the first image corresponding to the seventh area, and the third function is determined according to the fifth area on the third image and the sixth area on the first image corresponding to the fifth area;

0533: when the difference between the first loss information and the second loss information is within a preset threshold range, determining an average value of the first loss information and the second loss information as the obstacle avoidance reference information; or

0534: When the difference between the first loss information and the second loss information is outside a preset threshold range, determine that the one with larger depth information corresponding to the first loss information and the second loss information is the obstacle avoidance reference information.

In certain embodiments, the information determination module is also used to determine the first loss information based on the first depth information and the second function, and to determine the second loss information based on the second depth information and the fourth function, and to determine the average value of the first loss information and the second loss information as the obstacle avoidance reference information when the difference between the first loss information and the second loss information is within a preset threshold range, and to determine the larger one of the first loss information and the second loss information corresponding to the depth information as the obstacle avoidance reference information when the difference between the first loss information and the second loss information is outside the preset threshold range.

In certain embodiments, the processor is further used to determine first loss information based on the first depth information and the second function, and to determine second loss information based on the second depth information and a fourth function, and to determine, when the difference between the first loss information and the second loss information is within a preset threshold range, an average value of the first loss information and the second loss information as obstacle avoidance reference information, and to determine, when the difference between the first loss information and the second loss information is outside a preset threshold range, the one with the larger depth information corresponding to the first loss information and the second loss information as the obstacle avoidance reference information.

Specifically, in the case where the depth information Depth1 under infrared light and the depth information Depth2 under visible light are determined in the above embodiment, the method of determining the depth information Depth* according to the above data is described below.

First, based on the depth information Depth1 under infrared light that has been determined in the above embodiment, a loss depth information Loss1 (Depth1) (corresponding to the first loss information) is obtained by calculating the input side data of the loss function Loss1 determined in the process of transforming the RGB image according to the IR image.

Next, based on the depth information Depth2 under visible light that has been determined in the above embodiment, a loss depth information Loss2 (Depth2) (corresponding to the second loss information) is obtained by calculation as the input side data of the loss function Loss2 determined in the process of transforming the IR image according to the RGB image.

Then, the difference between Loss1 (Depth1) and Loss2 (Depth2) is calculated, and the obtained difference is compared with the preset loss information threshold range, and the final value of Depth* is determined according to the above comparison result.

For example, if the difference between Loss1(Depth1) and Loss2(Depth2) is within the above threshold range, it means that the accuracy of Depth1 and Depth2 meets the current obstacle avoidance requirements. In order to ensure the reliability of Depth*, Depth* can be taken as the average value of Loss1(Depth1) and Loss2(Depth2).

In addition, if the difference between Loss1(Depth1) and Loss2(Depth2) is outside the above threshold range, it means that at least one of Depth1 and Depth2 has an accuracy problem. In order to ensure the reliability of Depth*, Depth* takes the larger one of Loss1(Depth1) and Loss2(Depth2) to increase the distance between the mobile robot and the target object as much as possible during the obstacle avoidance process to ensure the safety of the obstacle avoidance process.

It should be noted that the specific execution order of the above steps of determining Loss1 (Depth1) and Loss2 (Depth2) can be swapped according to actual conditions or performed simultaneously. This embodiment describes only an exemplary step execution order and is not a limitation on the execution order.

Thus, the embodiments of the present application also provide a method for specifically determining obstacle avoidance reference information based on two sets of depth information.

The mobile robots involved in the embodiments of the present application refer to mechanical equipment designed for automatically performing work, including ground, air, water and underwater mobile robots, etc., and their mobile mechanisms include wheeled, tracked, footed, hybrid, special, etc. Mobile robots can include indoor mobile robots and outdoor mobile robots according to the working environment, wheeled mobile robots, walking mobile robots, snake-like mobile robots, tracked mobile robots, etc. according to the movement mode, and cleaning robots, disabled assistance robots, service robots, military robots, medical robots, etc. according to functions and uses.

Taking a cleaning robot as an example, it includes but is not limited to: a vacuum cleaner, a floor scrubber, a vacuum and water suction machine, a sweeper, a mop, a sweeper and mop all-in-one machine, etc. For ease of description, the mobile robot in the embodiment of the present application is described as a sweeper and mop all-in-one machine.

Fig. 11 is a schematic block diagram of a mobile robot in one embodiment. The mobile robot includes a robot body, a drive motor 102, a sensor unit 103, a controller 104, a cleaning member 105, a walking unit 106, a memory 107, a communication unit 108, an interaction unit 109, an energy storage unit 110, and the like.

The sensor unit 103 provided on the main body of the mobile robot may include at least one of the following sensors: a radar sensor (such as a laser radar), a visual sensor (such as an RGB camera), a collision sensor, a distance sensor, a drop sensor, a counter, a gyroscope, etc. For example, the laser radar is provided on the top or the side of the main body of the mobile robot. When working, it can obtain the surrounding environment information, such as the distance and angle of the obstacle relative to the laser radar. In addition, the laser radar can also be replaced by a visual sensor such as a camera. By analyzing the obstacle in the image taken by the camera, the distance and angle of the obstacle relative to the camera can also be obtained. The collision sensor, for example, includes a collision shell and a trigger sensor. When the mobile robot collides with the obstacle through the collision shell, the collision shell moves toward the inside of the mobile robot and compresses the elastic buffer to play a buffering role. After the collision shell moves a certain distance into the inside of the mobile robot, the collision shell contacts the trigger sensor, and the trigger sensor is triggered to generate a signal, which can be sent to the controller 104 in the main body of the mobile robot for processing. After colliding with the obstacle, the mobile robot moves away from the obstacle, and under the action of the elastic buffer, the collision shell moves back to its original position. The distance sensor may specifically be an infrared detection sensor, which can be used to detect the distance from the obstacle to the distance sensor. The distance sensor may be arranged on the side of the mobile robot body, so that the distance value from the obstacle located near the side of the mobile robot to the distance sensor can be measured by the distance sensor. The distance sensor may also be an ultrasonic distance sensor, a laser distance sensor or a depth sensor, etc. The drop sensor is arranged at the bottom edge of the mobile robot body. When the mobile robot moves to the edge of the ground, the drop sensor can detect that the mobile robot is at risk of falling from a height, thereby executing a corresponding anti-fall response, such as the mobile robot stopping moving, or moving in a direction away from the falling position, etc. A counter and a gyroscope are also arranged inside the mobile robot body. The counter is used to detect the length of the distance moved by the mobile robot. The gyroscope is used to detect the angle of rotation of the mobile robot, so as to determine the direction of the mobile robot.

The controller 104 is disposed inside the main body of the mobile robot, and is used to control the mobile robot to perform specific operations. The controller 104 may be, for example, a central processing unit (CPU) or a microprocessor. As shown in FIG11 , the controller 104 is electrically connected to components such as an energy storage unit 110, a memory 107, a drive motor 102, a walking unit 106, a sensor unit 103, an interaction unit 109, and a cleaning member 105 to control these components.

The cleaning member 105 can be used to clean the floor, and the number of the cleaning members 105 can be one or more. The cleaning member 105 includes, for example, a mop. The mop includes, for example, at least one of the following: a rotating mop, a flat mop, a roller mop, a crawler mop, etc., but is certainly not limited thereto. The mop is disposed at the bottom of the mobile robot body, and specifically can be at the rear of the bottom of the mobile robot body. Taking the cleaning member as a rotating mop as an example, a drive motor 102 is provided inside the mobile robot body, two rotating shafts extend from the bottom of the mobile robot body, and the mop is sleeved on the rotating shaft. The drive motor 102 can drive the rotating shaft to rotate, so that the rotating shaft drives the mop to rotate. The cleaning member 105 can also be a side brush, a roller brush, etc., which are not limited here.

The walking unit 106 is a component related to the movement of the mobile robot, and the walking unit 106 includes, for example, a driving wheel and a universal wheel. The universal wheel and the driving wheel cooperate to realize the steering and movement of the mobile robot.

The memory 107 is arranged on the main body of the mobile robot, and a program is stored in the memory 107, and the corresponding operation is realized when the program is executed by the controller 104. The memory 107 is also used to store parameters used by the mobile robot. The memory 107 includes but is not limited to a disk memory, a compact disc read-only memory (CD-ROM), an optical memory, etc.

The communication unit 108 is arranged on the main body of the mobile robot, and is used to allow the mobile robot to communicate with external devices, such as a terminal or a base station, wherein the base station is a cleaning device used in conjunction with the mobile robot.

The interaction unit 109 is disposed on the main body of the mobile robot, and the user can interact with the mobile robot through the interaction unit 109. The interaction unit 109 includes, for example, at least one of a touch screen, a switch button, a speaker, etc. For example, the user can control the mobile robot to start or stop working by pressing the switch button.

The energy storage unit 110 is disposed inside the main body of the mobile robot, and the energy storage unit 110 is used to provide power to the mobile robot.

The main body of the mobile robot is also provided with a charging component, which is used to obtain power from an external device (such as a base station) to charge the energy storage unit 110 of the mobile robot.

It should be understood that the mobile robot described in FIG. 11 is only a specific example in the embodiment of the present application, and does not constitute a specific limitation on the mobile robot in the embodiment of the present application. The mobile robot in the embodiment of the present application may also be other specific implementations. In other implementations, the mobile robot may have more or fewer components than the mobile robot shown in FIG. 11; for example, the mobile robot may include a clean water chamber for storing clean water and/or a dirt holding portion for storing dirt, and the mobile robot may transport the clean water stored in the clean water chamber to the mop and/or the ground to wet the mop, and clean the ground based on the wetted mop, and the mobile robot may also collect dirt on the ground or sewage containing dirt into the dirt holding portion; the mobile robot may also transport the clean water stored in the clean water chamber to the mop to clean the mop, and the sewage containing dirt after cleaning the mop may also be transported to the dirt holding portion.

The computer-readable storage medium in the embodiments of the present application stores a computer program, and when the computer program is executed by one or more processors, the above method is implemented.

The computer-readable storage medium in the embodiment of the present application stores a computer program, and when the computer program is executed by one or more processors, the above method is implemented. The computer-readable storage medium may include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

In the description of this specification, the descriptions with reference to the terms "certain embodiments", "in an example", "exemplarily", etc., mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations to the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims (12)

1. A mobile robot control method, characterized in that the method comprises: Acquire a first image of the target object; Performing a first spectral transformation process on the first image to determine a second image corresponding to the first image; Acquiring a third image of the target object; Performing a second spectral transformation process on the third image to determine a fourth image corresponding to the third image, wherein the first image and the third image have different spectral types, the first image and the fourth image have the same spectral type, and the second image and the third image have the same spectral type; Determining obstacle avoidance reference information according to the first image and the fourth image, and the second image and the third image; The mobile robot is controlled to avoid the target object according to the obstacle avoidance reference information. 2. The method according to claim 1, characterized in that the method further comprises: Based on a laser device arranged on the mobile robot, a reference laser is emitted to the target object, wherein the first image and the fourth image are both formed by the target object reflecting the reference laser. 3. The method according to claim 1, wherein the first image is an infrared image, and the third image is a visible light image; The performing a first spectral transformation process on the first image to determine a second image corresponding to the first image includes: Determine a first function according to a first area on the first image and a second area on the third image corresponding to the first area, wherein the first function includes a first parameter, a second parameter, and a third parameter corresponding to three color dimensions of the visible light image; Determine a second function according to a third area on the first image, the first function, and a fourth area on the third image corresponding to the third area, wherein the second function includes a fourth parameter, a fifth parameter, and a sixth parameter corresponding to three color dimensions of the visible light image; The second image is determined according to the first function, the second function and the first image. 4. The method according to claim 3, characterized in that determining the first function according to the first area on the first image and the second area on the third image corresponding to the first area comprises: The first parameter, the second parameter, and the third parameter are determined according to the pixel value of the first area and the pixel value of the second area, wherein the first area and the second area have the same image feature. 5. The method according to claim 3, characterized in that determining the second function according to the third area on the first image, the first function, and a fourth area on the third image corresponding to the third area comprises: determining a first comparison image according to the third area and the first function, wherein the first comparison image is a visible light image; The fourth parameter, the fifth parameter and the sixth parameter are determined according to the difference between the first comparison image and the fourth region. 6. The method according to claim 1, characterized in that the first image is an infrared light image, and the third image is a visible light image; The performing a second spectral transformation process on the third image to determine a fourth image corresponding to the third image includes: determining a third function according to a fifth area on the third image and a sixth area on the first image corresponding to the fifth area, wherein the third function includes a seventh parameter, an eighth parameter, and a ninth parameter corresponding to three color dimensions of the visible light image; determining a fourth function according to a seventh area on the third image, the third function, and an eighth area on the first image corresponding to the seventh area, wherein the fourth function includes a tenth parameter, an eleventh parameter, and a twelfth parameter corresponding to three color dimensions of the visible light image; The fourth image is determined according to the third function, the fourth function and the third image. 7. The method according to claim 6, characterized in that the determining of the third function according to the fifth area on the third image and the sixth area on the first image corresponding to the fifth area comprises: The seventh parameter, the eighth parameter, and the ninth parameter are determined according to the pixel value of the fifth region and the pixel value of the sixth region, wherein the fifth region and the sixth region have the same image feature. 8. The method according to claim 6, wherein determining the fourth function according to the seventh area on the third image, the third function, and an eighth area on the first image corresponding to the seventh area comprises: determining a second comparison image according to the seventh area and the third function, wherein the second comparison image is an infrared light image; The tenth parameter, the eleventh parameter and the twelfth parameter are determined according to the difference between the second comparison image and the eighth region. 9. The method according to claim 1, wherein determining obstacle avoidance reference information according to the first image and the fourth image, and the second image and the third image, comprises: determining first depth information according to the first image and the fourth image; determining second depth information according to the second image and the third image; The obstacle avoidance reference information is determined according to the first depth information and the second depth information. 10. The method according to claim 9, wherein determining the obstacle avoidance reference information according to the first depth information and the second depth information comprises: determining first loss information according to the first depth information and a second function, wherein the second function is determined according to a third area on the first image, a first function, and a fourth area on the third image corresponding to the third area, and the first function is determined according to a first area on the first image and a second area on the third image corresponding to the first area; determining second loss information according to the second depth information and a fourth function, wherein the fourth function is determined according to a seventh area on the third image, a third function, and an eighth area on the first image corresponding to the seventh area, and the third function is determined according to a fifth area on the third image and a sixth area on the first image corresponding to the fifth area; When the difference between the first loss information and the second loss information is within a preset threshold range, determining an average value of the first loss information and the second loss information as the obstacle avoidance reference information; or When the difference between the first loss information and the second loss information is outside the preset threshold range, the one corresponding to the larger depth information of the first loss information and the second loss information is determined as the obstacle avoidance reference information. 11. A mobile robot, characterized in that the mobile robot includes a visible light camera device, an infrared light camera device and a laser device; the mobile robot also includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the method described in any one of claims 1 to 10 is implemented. 12. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 10 is implemented.