CN103612252A - Intelligent remote social adjuvant therapy robot for autism children - Google Patents

Abstract

The invention belongs to the field of remote control and artificial intelligence, and relates to an intelligent auxiliary rehabilitation treatment robot device for real-time social ability training for autistic children. The device includes a head motion driver (1), an arm motion driver (2), an equipment carrier (3), a master-slave wheeled chassis (4), and a housing module (7). The multi-joint joint drive method is adopted to improve the cooperative drive ability of the robot to each joint and solve the problem that the delay affects the interaction effect. Through the PWM bidirectional control technology, the attitude control of the robot in any direction is achieved, and the limbs that automatically avoid obstacles are achieved. The motion path planning function, aiming at the common emotional cognitive impairment in children, realizes the intelligent emotional interaction between human and robot and realizes the safe and reliable Home telerehabilitation treatment model.

Description

Intelligent Remote Social Assisted Therapy Robot for Children with Autism

technical field

The invention belongs to the field of remote control and artificial intelligence, and relates to an intelligent auxiliary rehabilitation treatment robot device and a system thereof for real-time social ability training for autistic children.

technical background

According to incomplete statistics, there are currently about 35 million autistic patients in the world, and the prevalence rate is as high as 94:1. According to a conservative estimate by the China Disabled Persons' Federation in 2009, there are 1.5 million autistic children aged 0 to 15 in my country, and 5 million children with autism aged 0 to 20, and the incidence rate is increasing year by year. Because the cause of autism is unknown, there is no effective treatment. In addition to drug intervention, the current training methods for autism generally use one-on-one behavior correction and simple rehabilitation games. Major problems such as long training period (most children need to participate in training for life), extremely high medical expenses, scarcity of medical personnel, uneven level of medical institutions, and poor auxiliary treatment tools have become increasingly prominent.

In view of the above outstanding problems, advanced information science methods such as control and artificial intelligence are combined with auxiliary treatment to develop a remote social auxiliary treatment robot for children with autism. And therapy provides new tools. As a comprehensive and credible interactive system, the robot is applied to schools and family education, serving doctors, teachers, parents and autistic children with special needs, and providing autistic children with cognitive and emotional abilities through humanized social assistance therapy The recovery brought dawn.

Contents of the invention

In order to timely and effectively improve the social interaction ability of children with autism, provide advanced rehabilitation tools for children with autism. One of the purposes of the present invention is to provide an auxiliary treatment robot device, which realizes a social auxiliary treatment mode based on multiple interactive means through multi-joint drive, obstacle avoidance, path planning, intelligent information fusion and other control technologies; at the same time, it provides network Isolation management, adaptive data filtering and other network security technologies are used to realize the independent filtering and active defense functions of threat information in the wide area network, thereby providing a strong guarantee for the privacy of children and the safety of robot systems; Analytical emotion capture and affective computing technology to achieve coordinated and synchronized human-computer interaction based on the emotional characteristics of children, so as to effectively cultivate autistic children's social interaction and emotional communication skills.

The specific plan is as follows:

A remote assisted therapy robot device, including a head motion driver (1), an arm motion driver (2), a device carrier (3), a master-slave wheeled chassis (4), and a housing module (7), the robot The head movement driver (1) of the device is mounted on the top of the equipment carrier (3); the arm drivers (2) are respectively mounted on both sides of the equipment carrier (3); the chassis (4) is installed on the bottom of the carrier (3).

Preferably, the head motion drivers (1) are divided into two groups, and the two groups are vertically coupled to each other, and each group of head motion drivers (1) includes a clutch (101), a reducer (102), a DC motor (103) and its DC motor control drive mechanism (104); the DC motor control drive mechanism is an annular DC motor control board, and the DC motor control board is equipped with 5 photoelectric sensors (105) and 2 micro switches (106), and the photoelectric sensor (105) and the micro switch (106) are evenly distributed on the outer edge of the semicircle of one side of the DC motor control board (104) at an angle of 30 degrees, and the angle between the two micro switches (106) is 180 degrees .

Preferably, the arm driver (2) is two sets of multi-motor coordinated drive devices installed on the left and right sides of the carrier (3); wherein each arm driver (2) includes four step-by-step motion joints (107), 1 piece of arm motor control board (120), 1 piece of shoulder joint support (108), 1 piece of large arm support (109), 1 piece of forearm support (110), 1 piece of hand support (111), 1 piece of 3 million Pixel COMS camera (118).

Preferably, each stepping motion joint (107) is composed of a stepping motor and its reducer; the shoulder joint is composed of two stepping motion joints (107) vertically coupled, and the two stepping motion joints (107) Control the rotational degree of freedom and the vertical swing degree of freedom separately.

Preferably, the device carrier (3) includes a vertical support (112), a foldable support (113), a pull-out frame (114) and a drawable rectangular Partition (115); the vertical bracket (112) is used to install the head motion driver (1), the foldable bracket (113) is used to install the touch interactive screen, and the rectangular partition (115) is used to place the arm motor The control board (120) and the industrial control computer (8); the head movement driver (1) and the arm driver (2) use the RS485 bus as the physical transmission basis, and exchange information with the industrial control computer (8) through the ModBus protocol.

Preferably, the main-slave wheel chassis (4) includes 2 main driving wheels (116), 1 omnidirectional driven wheel (117) and a chassis carrier (122); the main driving wheel (116) The DC motor controls the driving mechanism (104) in common with the head motion driver (1).

Preferably, the shell module (7) is made of hard PVC material, and the shell module (7) is equipped with 5 flexible touch sensors (124), including 1 on the top of the head, 2 on the ear, and 2 on the shoulder At the same time, the shell module (7) is equipped with 11 sets of ultrasonic distance measuring sensors (125), including 4 sets at the front, 3 sets at the rear, and 2 sets at each side.

Preferably, a multi-sensor control and information feedback module (5), a network security module (6) are also installed on the rectangular partition (115),

Wherein, the multi-sensor control and information feedback module (5) is a sensor information sending and receiving control module, which exchanges information with each sensor, and uses the RS485 bus as the physical transmission basis to perform information interaction with the computer motherboard (8) through the ModBus protocol, The sensor is a flexible touch sensor and an ultrasonic ranging sensor;

The network security module (6) is composed of two parts: a physical isolation module and a data filtering module; wherein, the physical isolation module includes four 6N317 photoelectric isolation chips and two ARM processors.

Preferably, the remote assisted treatment robot device includes a video acquisition module, a voice interaction module, an LED output module (128), an LCD display (129) and a TC touch (130) module; wherein the video acquisition module includes a A CCD high-speed industrial camera (119) on the top of the frame (114) is used to collect expressions and micro-expressions of children; the voice interaction module includes a built-in wireless microphone (126) installed on the top of the pull-out frame (114). ), robot head and 3 loudspeakers (127) behind it, speech recognition and synthesis unit; LED output module (128) includes the LED lamp of robot head and hand and its LED control circuit; LCD display module (129 ) and the TC touch module (130) are respectively an LCD display installed on the head movement driver (1) and a touch interactive screen installed on the foldable support (113).

An intelligent social system for children with autism, including the above-mentioned remote assisted therapy robot device, a social ability training subsystem (9), an intelligent emotion analysis subsystem (10), a scale evaluation and database subsystem (11); The video acquisition module, voice interaction module, LED output module (128), LCD display (129) and TC touch (130) modules are mounted on the RS485 bus, and communicate with the social ability training subsystem through the ModBus protocol; intelligent emotion The analysis subsystem adopts the non-invasive EEG monitoring module to collect the EEG information of the children in real time, and wirelessly transmits it to the intelligent emotion analysis subsystem through Bluetooth for analysis.

Advantage and positive effect of the present invention are:

1) Through the information interaction between the RS485-ModBus protocol conversion module and the industrial control computer, the multi-joint joint drive mode is realized, so that the robot has real-time motion joint drive capability; the introduction of the F_Direction reference vector reduces the deviation between random nodes and the optimal path, This effectively shortens the time-consuming path planning and improves the real-time performance and accuracy of robot movement; combined with real-time image processing technology, it realizes the limb movement path planning function that can automatically avoid obstacles; The combination method controls the movement speed of the limbs and calculates the movement angle and distance to realize the attitude control of the robot in any direction with precision, so that the robot can accurately complete the interactive commands in real time during the human-computer interaction process, effectively improving the success rate of the interaction.

2) The device is equipped with flexible touch sensors, ultrasonic ranging sensors, photoelectric sensors and other sensing devices, image collectors such as COMS cameras and CCD cameras, and is equipped with various information output devices such as LEDs and LCDs. The social and emotional interaction in the system provides an intelligent basis; various types of I/O devices are coordinated and managed through multi-sensor control and information feedback modules, each module is mounted on the RS485 bus, and directly communicates with the RS485-ModBus protocol conversion module Control center communication, this distributed information fusion technology enables the system to have efficient communication management capabilities, and at the same time enhances fault tolerance and facilitates tailoring and upgrading of equipment.

3) According to the opinions of medical experts, six progressive training modes with step-by-step difficulty are designed to provide practical and effective treatment plans for children of different ages and levels; multi-sensor, voice, image, TC touch and other means are used to fully Fangfang conducts social interaction training, which greatly increases the ways of emotional interaction, and has a positive impact on the effect of interactive training while improving children's participation interest; combining high-speed industrial cameras with facial expression and micro-expression recognition and analysis technology and brain The combination of electronic monitoring technology provides a reliable engineering and physiological basis for the judgment of the emotional state of children and the phased results of adjuvant therapy; through the emotion transfer algorithm, the personalized human-computer interaction process is realized, and children are trained to generate empathy and social interaction Ability; the use of information management methods based on large databases is conducive to safety management, data collection and disease analysis, and can also provide reference and basis for future treatment.

4) The system has two modes of autonomous training and remote interactive therapy. The autonomous training mode can enable children to carry out social ability training anytime and anywhere in the environment such as the family, and organically integrates treatment and entertainment. The remote interactive therapy mode can make patients Children can enjoy the doctor's rehabilitation treatment guidance without leaving home, and doctors can also keep abreast of and master the children's rehabilitation progress while providing online medical care; in the independent training mode, the training data is transmitted to the remote server in one direction through the safety isolation module , which can not only ensure the real-time communication, but also realize the function of hardware to block the threat of external network information; in the remote interactive treatment mode, the robot system adopts the adaptive data filtering algorithm based on association rule mining to effectively block the dangerous information on the external network and prevent privacy leakage , Guarantee the safe operation of the interactive system; the proposal of this treatment mode can effectively save medical resources and vigorously promote the development of family telerehabilitation treatment.

Description of drawings

Fig. 1 is a diagram of the relationship between various functional modules of the robot of the present invention.

Fig. 2 is a mechanical structure diagram of the head of the robot of the present invention.

Fig. 3 is a schematic diagram of the electric connection of the core circuit of the robot head and chassis motion control of the present invention.

Fig. 4 is a perspective view of the mechanical structure of the robot arm of the present invention.

Fig. 5 is a schematic diagram of the electrical connection of the core circuit of the robot arm motion control of the present invention.

Fig. 6 is a top view of the "convex"-shaped chassis of the robot of the present invention.

Fig. 7 is a structural diagram of the robot equipment carrier of the present invention.

Fig. 8 is a structural diagram of a robot folding touch screen bracket of the present invention.

Fig. 9 is an overall mechanical structure diagram of the robot of the present invention.

Fig. 10 is an effect drawing of the shell of the robot of the present invention.

Fig. 11 is a schematic diagram of the electrical connection of the robot equipment control/communication core circuit of the present invention.

Fig. 12 is a schematic diagram of electrical connection of the robot network security control core of the present invention.

Fig. 13 is a flow chart of main body control of robot intelligent remote social assisted therapy of the present invention.

Detailed ways

Further description will be made below in conjunction with accompanying drawings and examples.

Fig. 1 is a diagram of the relationship between various functional modules of the robot of the present invention. The present invention provides a safe and reliable human-computer intelligent interaction technology in remote mode by applying a multi-level modular management structure, realizes real-time data collection and processing, intelligent response and remote auxiliary treatment functions for autistic children, and solves the existing problems Lack of resources for socially assisted therapy tools, insufficient levels of emotional interaction, and limited remote security.

As shown in Figure 2, there are two groups of head motion drivers (1) of the robot, which are vertically coupled to each other. Each group of head motion drivers (1) includes mutually independent clutches (101), reducers (102), DC motors (103) and its control driving mechanism (104), wherein each group of head motion drivers (1) is responsible for one degree of freedom, one group is responsible for the nodding motion of the robot, and one group is responsible for the head shaking motion. The control drive mechanism (104) of the DC motor is a DC motor control board, which is in the shape of a ring and is vertically embedded on the outside of the reducer (102). Distributed on the outer edge of the semicircle on one side of the control board (104), the angle between the two micro switches (106) is 180 degrees, and the distance from the micro switch (106) to the center of the control board (104) is 5 cm , the distance from the photoelectric sensor (105) to the center of the control panel (104) is 6 cm. The head movement driver (1) uses the STC12C5A16AD chip as the core controller, that is, the DC motor control board (104). Its control circuit is shown in Figure 3, and the DC motor is controlled by combining PWM bidirectional control with sensors and delay calculations Rotation angle, for example, control the rotation angle through the DC motor (103) to control the delay (angular velocity × time delay) within each 30°, and the maximum rotation of the head can be set through the micro switch (106) with limit function Angle physical limit, to achieve precise positioning of the motor movement angle.

As shown in Fig. 4, the arm driver (2) of the robot is driven by three motion mechanisms of the shoulder joint, elbow joint, and wrist joint to produce actions. The arm driver (2) is installed on the left and right sides of the carrier (3) respectively, and each The group of arm drivers (2) includes 4 step-by-step motion joints (107), 1 arm motor control board (120), 1 shoulder joint bracket (108), 1 big arm bracket (109), and 1 forearm A bracket (110), a hand bracket (111), a 3 million pixel COMS camera (118). Wherein, each stepping motion joint (107) is composed of a stepping motor and its reducer; the elbow joint and the wrist joint respectively include a stepping motion joint (107), and the shoulder joint consists of two stepping motion joints ( 107) Vertical coupling structure, two step-by-step kinematic joints (107) respectively control two degrees of freedom, one degree of freedom for rotation, and one degree of freedom for swinging up and down, which can realize the functions of 360-degree omnidirectional rotation of the arm and 180-degree vertical swing. The arm driver (2) adopts the TB6560AHQ chip as the core controller, that is, the arm motor control board (120). The rotation axis and the other three stepping motion joints (107) (the stepping motion joint (107) in the shoulder joint responsible for swinging up and down, and the two stepping motion joints (107) corresponding to the elbow joint and wrist joint respectively )) Cooperative control, the robot arm can flexibly point to any direction, and accurately complete the control commands issued by the system in real time, with a control error of less than 0.5 degrees. In addition, a 3-megapixel COMS camera (118) is installed on both sides of the robot to collect real-time images of the arm pose and the surrounding environment, upload them in real time through the USB communication interface, and establish a three-dimensional coordinate system to realize the best path for arm movements. Planning and calculation, and introducing the F_Direction reference vector to reduce the deviation between random nodes and the optimal path, thereby shortening the time-consuming path planning, improving the real-time performance and accuracy of the robot's movement, and at the same time realizing the obstacle avoidance function, effectively avoiding the robot's limb movement The accidental injury caused by the robot makes the interaction process between the child and the robot safer.

As shown in Figure 6, the robot adopts a "convex"-shaped main-slave wheel chassis design, and the main-slave wheel chassis (4) includes 2 main driving wheels (116) and 1 omnidirectional driven wheel (117) and a "convex" shaped chassis carrier (122). There is a battery installation position (123) on the "convex" shaped chassis carrier. The main drive wheel (116) is driven by an independent DC motor, and shares the core controller with the head movement driver (1), that is, the DC motor control board (104) (Fig. 2), and drives the DC motor to rotate and move through PWM bidirectional control. The speed is flexible and adjustable, and it can complete the fast action response of multiple speed levels such as variable/variable speed, straight/curve movement, etc.

The head motion driver (1) of the robot is mounted on the top of the carrier (3); the arm drivers (2) are respectively mounted on both sides of the carrier; the chassis (4) is installed at the bottom of the carrier (3). The structure diagram of the equipment carrier (3) of the robot of the present invention is shown in Fig. 7. The equipment carrier (3) includes a vertical support (112), a folding support (113), a pull 2 drawable rectangular partitions (115) in the frame (114) and the pull-out frame (114). Wherein, the vertical bracket (112) is used for installing the head motion driver, and the folding bracket (113) is used for installing the touch interactive screen. The folding bracket (113) for installing the touch-type interactive screen adopts a three-fold structure design, as shown in Figure 8, an omnidirectional rotation axis is set at each folding position, that is, three 360-degree omnidirectional rotation axes are included, and the touch screen can be Adjust to a suitable height and angle for operation, so that the design of the interactive device is more humanized, and it can also achieve the purpose of saving storage space; ), the three-dimensional application of the space is realized by installing two partitions (115) on the upper and lower slide rails, the pull-out frame (114) is used to install the arm driver and support the entire robot device, the upper rectangular partition (115 ) is used to place the industrial control computer (8), and the lower rectangular partition (115) is used to place the arm drive board (ie, the arm motor control board (120)), the multi-sensor control and information feedback module (5), and the network security module ( 6), each control board installed on the rectangular partition (115), including industrial control computer (8), arm motor control board (120), multi-sensor control machine information feedback module (5), network security module (6) , It is erected with an isolation column, which is convenient for system maintenance and hardware upgrade. This constitutes the overall mechanical structure of the robot as shown in FIG. 9 , and FIG. 10 is a rendering of the robot device after the shell is installed. The shell module (7) is made of hard PVC material and has a cartoon shape. This module is equipped with a total of 5 flexible touch sensors (124), including 1 on the top of the head, 2 on the ears, and 2 on the shoulders. At the same time, it is equipped with 11 sets of ultrasonic distance measuring sensors (125), of which 4 sets on the front, 3 sets in the rear, 2 sets on each side.

The robot's head motion driver (1) and arm driver (2) are mounted on the RS485 bus to realize multi-joint joint drive, and the RS485-ModBus protocol conversion module communicates with the industrial control computer (8) to perform information interaction, thereby improving the robot The driving ability of each joint joint solves the problem that the delay affects the interaction effect. The device applies path planning and obstacle avoidance technology to the motion control of the robot, and at the same time introduces the F_Direction reference vector to reduce the deviation between random nodes and the optimal path, thereby effectively shortening the time-consuming path planning and improving the motion accuracy of the device. Timeliness and accuracy. The robot's head joints and chassis driving wheels are driven by DC motors, and the movement angle and distance are measured by combining PWM bidirectional control technology with sensors and delay calculations; the arm joints are driven by stepping motors, and the image processing technology and the above path Combining planning and obstacle avoidance technology, the control error is less than 0.5 degrees, and realizes the posture control of the robot in any direction and the limb movement path planning function that can automatically avoid obstacles. This function not only improves the motion controllability of the device, but also effectively avoids obstacles. The accidental injury caused by the movement of robot limbs makes the interaction process between children and robots safer.

In the present invention, the multi-sensor control and information feedback module (5) is a transceiver control module for all sensor information, and the multi-sensor control and information feedback module (5) exchanges information with each sensor, and converts the module and industrial control through the RS485-ModBus protocol The computer (8) performs information interaction, and the sensors are flexible touch sensors (124) and ultrasonic distance measuring sensors (125).

The network security module (6) is composed of two parts: a physical isolation module and a software filtering module. The physical isolation module consists of four 6N137 photoelectric isolation chips and two ARM processors. In the autonomous training mode, four 6N137 photoelectric isolation chips with a conversion rate of up to 10Mb/s can realize the function of hardware blocking external network information interference while ensuring the upload speed of system training information; an ARM processor is used for the security module and the robot system. For communication and information control, another ARM processor is used for data format conversion and data caching, packs data according to the Ethernet protocol specification and completes the information transmission and communication control functions of the external network. In the remote interactive treatment mode, high-frequency item sets are extracted from the dynamically updated network security database, data association rules are established hierarchically, and genetic algorithms are used to adjust association parameters in real time to derive new association rules, thereby realizing adaptive data filtering functions to ensure the safe operation of the interactive system, thereby realizing a safe and reliable home telerehabilitation treatment mode.

The present invention collects and analyzes multi-sensor information, specifically including:

1) The facial expression and micro-expression acquisition module collects facial expressions of children through the F-033BL CCD high-speed industrial camera (119) installed on the top of the pull-out frame. The resolution of the camera is 656×494, and the frame rate is as high as 58 frames/ Seconds, therefore, micro-expression key frames can be captured while collecting images of children's facial expressions in real time. The collected video will be transmitted to the intelligent emotion analysis subsystem in real time through the 1394a interface. After image processing such as feature extraction and expression recognition, the real emotional change process of the child can be more accurately understood, so that the robot can produce appropriate emotional interaction behaviors. It also provides an objective analysis basis for the emotional analysis and social training planning of children with autism.

2) The non-invasive EEG monitoring module can collect the EEG information of children in real time through non-invasive methods, and wirelessly transmit it to the intelligent emotion analysis subsystem through Bluetooth for analysis, and compare it with the expression and micro-expression image processing results Yes, comprehensively generate the emotional judgment results of the children, and provide a reliable physiological basis for the adjuvant treatment of children with autism and the professional guidance of doctors.

3) 5 flexible touch sensors are distributed on the top of the head, ears and shoulders of the robot device (can be increased or decreased arbitrarily according to specific needs). The flexible touch sensors are based on roll-type metal grid technology, and the underlying fine-line metal is used to achieve narrow edges The curved surface of the arc is applied, and its thinness and light weight are conducive to the installation of the streamlined surface of the robot. The sensor provides a richer touch experience for children, makes the anthropomorphic effect of the robot more realistic, and enhances the interaction during the interactive training process. Perceptual ability, which can improve the affinity of training.

4) 11 sets of ultrasonic ranging sensors are distributed on the outer edge of the robot chassis movement mechanism, and the distance of obstacles around the robot is judged by analyzing the distance data collected in parallel, with an error of less than 0.5 cm. Provide a suitable operating distance for the interaction process of children, ensure that the expression and micro-expression collection module collects the facial images of the interactor at the best position, and realize the anti-collision protection function of the robot through the detection of environmental obstacles.

The RS232-RS485 conversion and RS485-ModBus protocol conversion modules involved in the present invention adopt the control/communication circuit design with STM32F103RC as the core, as shown in FIG. 11 . Among them, STM32F103RC is a 64-pin microcontroller using a 32-bit Thumb2 reduced instruction set system. It is one of the embedded chips on the market that integrates high performance, low power consumption, and strong real-time performance. The microcontroller has a Cortex-M3 core, a built-in high-speed memory up to 256K bytes, and its I/O turnover speed can reach 18MHz, which can fully meet the needs of one-to-many data collection, device control, protocol conversion and real-time communication. The distributed equipment adopts the bus-type control method, and its transmission rate can reach up to 10Mbit/s, allowing 128 transceiver devices to be connected, which greatly improves the equipment expansion capability of the robot device. The circuit design has a signal protection circuit, so that the system has a good performance. Lightning protection, anti-surge and over-current protection functions make the system more stable.

The robot uses a photoelectric isolation chip combined with dual ARM processors to implement isolated management from the wide area network to the local area network. The network upload mode for independent training and the remote interactive treatment network transmission mode based on the adaptive data filtering algorithm based on association rule mining. Two kinds of network security control In this way, the ability to protect the privacy of children and the safety of the robot system can be improved, and a safe and reliable family remote rehabilitation treatment mode can be realized. In the autonomous training mode, as shown in Figure 12, four 6N137 photoelectric isolation chips with a conversion rate of up to 10Mb/s can realize the function of hardware blocking external network information interference while ensuring the upload speed of system training information, so as to ensure training data It can be uploaded to the server in real time and prevent the system from being attacked by external network viruses in the autonomous training mode; two ARM946E-S processors have a cache that can be dynamically allocated, which are used for communication and information control between the security module and the robot system. And the conversion of LAN and Ethernet data transmission protocols to ensure that the data is packaged according to the Ethernet protocol specifications and complete the information transmission and communication control functions of the external network. In the remote interactive treatment mode, high-frequency item sets are extracted from the dynamically updated network security database, data association rules are established hierarchically, and genetic algorithms are used to adjust association parameters in real time to derive new association rules, so that through self-adaptive data The filtering algorithm effectively shields dangerous information from the external network to ensure the safe operation of the interactive system.

The main control flow of the robot intelligent remote social assisted therapy of the present invention is shown in FIG. 13 . The control center of the intelligent social system for children with autism is a smart industrial control computer (equipped with Windows 7 operating system) produced by North China Industrial Controls. The auxiliary I/O structure includes: video acquisition module, EEG acquisition module, non-specific human speech recognition module, speech synthesis module, LED multi-function output module, LCD image display module and TC touch interaction module, etc.

The social ability training subsystem is the front desk that directly interacts with the children; the intelligent emotion analysis uses the expression images collected by the front desk to map the emotional state based on the continuous state space, and combines the results of the EEG analysis in the three-dimensional space to apply the theory based on psychological energy and probability. The emotional transfer algorithm is used for emotional calculation, and the results are converted into control instructions and sent to the social ability training subsystem to generate humanized interactive responses; the scale evaluation and database subsystem can not only complete the process data and expression images during the interaction process , EEG signal and intelligent emotional analysis results archiving and storage function, and can also realize the online filling function of five scales of ABC, ASSQ, CARS, social response scale, and Kresch scale, which is used to manage the third party of medical staff and parents The evaluation results provide an effective basis for future data mining and efficacy evaluation.

Specifically, the hardware equipment of the social ability training subsystem includes a video acquisition module, a voice interaction module, an LED output module (128), an LCD display (129) and a TC touch (130) module. The video acquisition module collects facial expressions and micro-expressions of children through a CCD high-speed industrial camera (119) installed on the top of the pull-out frame, and transmits the image information to the intelligent emotion analysis subsystem in real time. The voice interaction module includes a built-in wireless microphone (126) installed on the top of the pull-out frame, three loudspeakers (127) at the headphone position and the back of the robot, and a voice recognition and synthesis unit. The speech synthesis unit is equipped with three sound modes of male voice, female voice, and children's voice to choose from, and has surround sound effects; the LED output module (128) mainly includes colorful LED indicators on the head and hands and its control circuit. This attracts the attention of children, and cooperates to complete related system functions such as joint attention. The LCD display module (129) and the TC touch module (130) are the core of the interactive operation, and are respectively the LCD display installed on the head movement driver (1), and the touch-sensitive touch module installed on the folding bracket (113). Interactive screen, the LCD display module uses a 5.7-inch (4:3) industrial-grade high-definition LCD screen, and the TC touch part uses a 7-inch (16:9) industrial-grade high-definition touch screen to ensure that the system has clear picture quality, good response speed and Can work in harsher environments. The hardware devices involved in each module are mounted on the RS485 bus through the RS232 to RS485 conversion board, and communicate with the social ability training subsystem through the RS485-ModBus protocol conversion module.

The intelligent emotion analysis subsystem adopts the non-invasive EEG monitoring module to collect the EEG information of the children in real time, and wirelessly transmits it to the intelligent emotion analysis subsystem through Bluetooth for analysis, providing a reliable basis for the auxiliary treatment of children with autism . At the same time, the emotional state mapping based on the continuous state space is carried out on the facial expression images transmitted from the social ability training subsystem, combined with the EEG analysis results, the emotional calculation is performed in the three-dimensional space using the emotional transfer algorithm based on psychological energy and probability theory, and the results are Converted into control instructions (the specific process is expression image preprocessing → 3D gradient projection (micro-expression key frame capture) → Gabor feature extraction → LBP feature dimensionality reduction → nearest neighbor classification based on gradient magnitude weighting), and finally give the control instruction to The remote assisted treatment robot device can realize the response of interactive expressions and actions, complete the personalized human-computer interaction process, and train children to develop empathy and social skills.

The scale evaluation and database subsystem uses SQL Server 2012 as the data management platform, which can be divided into four modules: patient information management, medical staff information management, scale evaluation, and interactive information recording. The child information management module mainly manages basic information such as the name, age, and gender of the child; the medical staff information management module is mainly used to set access rights according to different levels to effectively protect the privacy of the child; the scale evaluation module has ABC, ASSQ The online filling function of the five scales, CARS, Social Response Scale, and Klinefelter Scale, is used to manage the third-party evaluation results of medical staff and parents, and provide effective basis for future data mining and curative effect evaluation; the interactive information module is in After the child logs in, the reaction time, completion time, correct and error rate, repeated training times and many other process data of the interactive training will be synchronously entered into the database, so that the medical staff can fully understand and grasp the problems that the child presents during the training process , in order to carry out targeted counseling and treatment. The social ability training subsystem of the robot of the present invention is provided with six training modes: common attention, expression learning, expression test, expression imitation, expression jigsaw puzzle, and scene recognition. The following is a specific description based on its training mode:

1) Joint attention mode. The robot has various guidance methods such as voice guidance, voice + arm movement synchronous guidance, voice + arm movement + LED flashing synchronous guidance, voice + arm + head movement + LED flashing synchronous guidance, etc. face pose detection, to judge whether the child has completed the joint attention training, if completed, proceed to the next direction guidance, if not, increase the guidance intensity, and then judge the pose until the joint attention is completed or all the various guidance methods are used At the same time, data such as the response time of the child, the number of guidance repetitions, the completion of the action after each guidance, and the final completion of the action are submitted to the database for unified management.

2) Expression learning mode. In the facial expression learning part, there are preset facial expression pictures of six basic emotional states, and children can choose to learn by category or randomly according to their needs. The robot synchronously displays facial expression pictures through the LCD display and TC touch screen, and tells the emotional state contained in the facial expressions through voice and text prompts, gradually improving the cognitive and emotional deficits of children. At the same time, the database synchronously records the children's learned expressions and the corresponding facial expression video data during the learning process, providing a research basis for special education and professional guidance for doctors.

3) Expression test mode. According to the facial expression learning data, this module provides children with appropriate difficulty facial expression test questions by setting checkpoints. The specific test forms under the same difficulty are: choose one of three facial expression pictures, find the correct eye expression, and find the correct facial expression. Lip expressions. In order to improve children's learning interest and deepen memory, this test allows children to repeat the choice twice if they do not choose the correct one. At the same time, set up voice, LED lights and LCD cartoon expression rewards and encouragement mechanisms. Complete each question Finally, the correct answer is repeatedly displayed on the LCD screen to help children consolidate the expressions they have learned. Information such as the questions displayed during the test, the results of each selection, and the response time will be stored in the database synchronously. In addition, during the learning process, real-time monitoring of changes in children's expressions and EEG signals to analyze emotional changes, intelligently adjust the difficulty of the test, and timely improve the human-computer relationship during the training process, providing a positive impact on test learning.

4) Expression imitation mode. According to the types of facial expressions learned, this module will provide children with voice guidance and explanations to match the facial expressions displayed on the LCD screen to help children complete the imitation of facial expressions. Through the intelligent emotion analysis subsystem, the extraction and analysis of the child's expression features are completed, and the imitation fit of the expression is judged, and further action guidance is provided while encouraging the child, so as to improve the child's expression ability. During the simulation process, the data will be uploaded to the database synchronously, so that the medical staff can keep abreast of the patient's training progress and provide a practical basis for the setting of subsequent treatment plans.

5) Emoji puzzle. According to the children's current learning progress, this module will screen facial expression pictures and form two training methods: equal amount puzzle and difference puzzle. Children can drag and drop pictures through the TC touch screen to combine them into the expressions required in the voice prompt. If the first puzzle is not successful, the LCD screen will display encouraging pictures, and the voice prompts the child to try again; if the second puzzle is still unsuccessful, the system will guide the child to watch the full picture of the correct expression displayed on the LCD screen for a short time in the form of voice , and let the child do the puzzle again. When the child completes the puzzle correctly, the system will judge the child's nervousness based on the current EEG signal, and intelligently select reward music to relieve the child's nervousness.

6) Scene cognition. This module classifies the pictures according to the scene complexity of the pictures, and sets up three types of expression selection questions: choose one from two, choose one from four, and choose one from six. According to the learning progress of the child, the system will comprehensively consider the difficulty of the scene and the difficulty of the question type Unlock different levels. Children can use voice to explain the content of the pictures and their inner feelings after seeing the pictures to choose the corresponding expression and emotion pictures. At the same time, the system analyzes the inner emotional changes of the children in real time through the camera and EEG monitoring module to give correct emotional guidance With the guidance of choice, the children can recognize the emotional state of the scene and integrate into the scene to generate appropriate empathy, so as to help the children gradually get rid of the lonely inner world and tend to a normal cognitive and emotional state.

Claims (10)

1. a long-range auxiliary Therapy robot device, it is characterized in that: described robot device comprises head movement driver (1), arm motion driver (2), equipment carrier (3), MS master-slave wheel undercarriage (4) and shell module (7), this robot device's head movement driver (1) is equipped on equipment carrier (3) top; Arm driver (2) is equipped on respectively equipment carrier (3) both sides; Chassis (4) is installed on carrier (3) bottom.

2. according to right 1, require described long-range auxiliary Therapy robot device, it is characterized in that: described head movement driver (1) is two groups, mutually vertical coupled between two groups, every group of head movement driver (1) comprises clutch (101), decelerator (102), direct current generator (103) and DC MOTOR CONTROL driving mechanism (104) thereof; Described DC MOTOR CONTROL driving mechanism is annular DC MOTOR CONTROL plate, 5 photoelectric sensors (105) and 2 microswitches (106) are housed on DC MOTOR CONTROL plate, photoelectric sensor (105) and microswitch (106) are uniformly distributed in the outward flange of semi-circumference in DC MOTOR CONTROL plate (104) one sides with 30 degree angles, wherein between 2 microswitches (106), be 180 degree angles.

3. according to right 1, require described long-range auxiliary Therapy robot device, it is characterized in that: described arm driver (2) is for being installed on the device of motor coordinated drive more than two groups of carrier (3) left and right sides; Wherein every group of arm driver (2) comprises 4 step-by-step movement movable joints (107), 1 block of arm Electric Machine Control plate (120), 1 shoulder joint support (108), 1 large arm support (109), 1 forearm bracket (110), 1 hand supports (111), 1 3,000,000 pixel COMS camera (118).

4. according to right 3, require described long-range auxiliary Therapy robot device, it is characterized in that: each step-by-step movement movable joint (107) consists of stepper motor and decelerator thereof; Shoulder joint is by the vertical coupled formation of two step-by-step movement movable joints (107), and two step-by-step movement movable joints (107) are controlled respectively rotary freedom and swung up and down the free degree.

5. according to right 1, require described long-range auxiliary Therapy robot device, it is characterized in that: described equipment carrier (3) comprise in 1 stand type support (112), 1 folding bracket (113), 1 drawing and pulling type framework (114) and drawing and pulling type framework can pull rectangular clapboard (115); Described stand type support (112) is for securement head motion driver (1), and folding bracket (113) is for touch interactive screen is installed, and rectangular clapboard (115) is for laying arm Electric Machine Control plate (120) and industrial control computer (8); Described head movement driver (1) and arm driver (2) be take RS485 bus as physical transfer basis, by ModBus agreement and industrial control computer (8), carry out information interaction.

6. according to right 1, require described long-range auxiliary Therapy robot device, it is characterized in that: described MS master-slave wheel undercarriage (4) comprises 2 main drive wheels (116), 1 omnidirectional's driven pulley (117) and a chassis carrier (122); Described main drive wheel (116) is controlled driving mechanism (104) with head movement driver (1) common DC motor.

7. according to right 1, require described long-range auxiliary Therapy robot device, it is characterized in that: described shell module (7) is made by hard PVC material, shell module (7) is equipped with (124) 5 of flexible touch sensors altogether, wherein, 2 of the crown 1 Ge, ears, 2 of shoulders, simultaneously shell module (7) is equipped with 11 groups of ultrasonic distance-measuring sensors (125), wherein, and anterior 4 groups, 3 groups, rear portion, each 2 groups of both sides.

8. according to right 7, require described long-range auxiliary Therapy robot device, it is characterized in that: multisensor control and information feedback module (5), network security module (6) are also installed on described rectangular clapboard (115), wherein, described multisensor is controlled and information feedback module (5) is sensor information transmitting-receiving control module, carry out information exchange with each sensor, and take RS485 bus as physical transfer basis, by ModBus agreement and computer motherboard (8), carry out information interaction, described sensor is flexible touch sensor and ultrasonic distance-measuring sensor; Described network security module (6) consists of Physical isolation gap module and data filtering module two parts; Wherein, Physical isolation gap module comprises 4 6N317 photoelectric isolated chips, 2 arm processors.

9. according to right 1, require described long-range auxiliary Therapy robot device, it is characterized in that: described long-range auxiliary Therapy robot device comprises that video acquisition module, interactive voice module, LED output module (128), LCD show (129) and TC touch-control (130) module; Wherein video acquisition module comprises 1 the CCD high-speed industrial video camera (119) that is installed on drawing and pulling type framework (114) top, for infant being expressed one's feelings and micro-expression collection; Interactive voice module comprises 1 built-in wireless microphone (126), robot head and 3 loudspeakers (127), speech recognition and the synthesis unit behind that is installed on drawing and pulling type framework (114) top; LED output module (128) comprises LED lamp and the LED control circuit thereof of robot head and hand; LCD display module (129) and TC touch-control module (130) are respectively the LCD displays being installed on head movement driver (1), and are installed on the touch interactive screen on folding bracket (113).

10. towards an intelligent social system for autistic children belong, it is characterized in that: comprise long-range auxiliary Therapy robot device as claimed in claim 9, Sociality training subsystem (9), intelligent mood analyzing subsystem (10), scale assessment and database subsystem (11); Described video acquisition module, interactive voice module, LED output module (128), LCD show that (129) and TC touch-control (130) module carry are on RS485 bus, by ModBus agreement and Sociality training subsystem communication; Intelligence mood analyzing subsystem adopts the brain electric information of non-intrusion type brain electricity monitoring module Real-time Collection infant, and resolves in intelligent mood analyzing subsystem by Bluetooth wireless transmission.

Images