Detailed Description
The following detailed structural or functional description is provided as an example only, and various changes and modifications may be made to the embodiments. Thus, the described embodiments should not be construed as being limited to the present disclosure, and should be construed as including all changes, equivalents and/or substitutions within the spirit and technical scope of the present disclosure.
As used herein, the singular also includes the plural unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Unless defined otherwise, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. When the embodiments are described with reference to the drawings, the same reference numerals denote the same elements, and a repetitive description thereof will be omitted.
Fig. 1 is a diagram illustrating an overview of a wearable device worn on a user's body according to an embodiment.
Referring to fig. 1, wearable device 100 may be a device that is worn on the body of user 110 to assist user 110 in walking, exercising, and/or working. In an embodiment, the wearable device 100 may be used to measure the physical ability (e.g., walking ability, exercise ability, or exercise posture) of the user 110. In an embodiment, the term "wearable device" may be replaced with "wearable robot", "walking assistance device" or "exercise assistance device". The user 110 may be a human or an animal, but the example is not limited thereto. The wearable device 100 may be worn on the body (e.g., lower body (leg, ankle, knee, etc.) or waist) of the user 110, and may apply external forces, such as an assisting force and/or a resistance force, to the body movement of the user 110. The assisting force may be a force assisting the body movement of the user 110, which is applied in the same direction as the direction of the body movement of the user 110. The resistance may be a force that resists body movement of the user 110, which is applied in a direction opposite to the direction of body movement of the user 110. The term "resistance" may also be referred to as "exercise load".
In an embodiment, wearable device 100 may operate in a walking assistance mode to assist the walking of user 110. In the walking assist mode, the wearable device 100 can assist the walking of the user 110 by applying an assist force to the body of the user 110, wherein the assist force is generated by the driving module 120 of the wearable device 100. The wearable device 100 may extend the walking capabilities of the user 110 by allowing the user 110 to walk independently or walk for a long time by providing the force required for the user 110 to walk. The wearable device 100 may also improve the walking of users with abnormal walking habits or gestures.
In an embodiment, wearable device 100 may operate in an exercise assisting mode to enhance the exercise effect on user 110. In the exercise assisting mode, the wearable device 100 may block the body movement of the user 110 or resist the body movement of the user 110 by applying the resistance generated by the driving module 120 of the wearable device 100 to the body of the user 110. When the wearable device 100 is a hip type wearable device worn on the waist (or pelvis) of the user 110 and the legs (e.g., thighs) of the user 110, the wearable device 100 worn on the legs of the user 110 may enhance the exercise effect on the legs of the user 110 by providing an exercise load to the leg exercises of the user 110. Alternatively, the wearable device 100 may apply an assisting force to the body of the user 110 to assist the exercise of the user 110. For example, when a disabled person or an elderly person wears the wearable device 100 to perform an exercise, the wearable device 100 may provide an assisting force to assist body movement during the exercise. In an embodiment, the wearable device 100 may provide resistance in one exercise session by providing a combination of assistance force and resistance at exercise sessions or time intervals, for example, providing assistance force in one exercise session, providing resistance in another exercise session.
In an embodiment, wearable device 100 may operate in a fitness measurement mode to measure the fitness of user 110. The wearable device 100 may measure movement information of the user using sensors (e.g., an angle sensor 125 and an Inertial Measurement Unit (IMU) 135) provided in the wearable device 100 while the user is walking or exercising, and may evaluate the physical ability of the user based on the measured movement information. For example, the gait index or exercise ability index (e.g., muscle strength, endurance, balance, or exercise movement) of the user 110 may be estimated from the movement information of the user 110 measured by the wearable device 100. The physical fitness measurement mode may include an exercise movement measurement mode that measures an exercise movement of the user.
In embodiments of the present disclosure, for ease of description, the wearable device 100 is described as an example of a hip-type wearable device, as shown in fig. 1, but embodiments are not limited thereto. As described above, the wearable device 100 may be worn on another body part (e.g., upper arm, lower arm, hand, lower leg, and foot) other than the waist and leg (particularly thigh), and the shape and configuration of the wearable device 100 may vary according to the body part on which the wearable device 100 is worn.
According to an embodiment, the wearable device 100 may include: a support frame (e.g., leg support frames 50 and 55 and lumbar support frame 20 of fig. 3) configured to support the body of user 110 when wearable device 100 is worn on the body of user 110; a sensor module (e.g., sensor module 520 of fig. 5 a) configured to obtain sensor data including motion information about body motions (e.g., leg motions and upper body motions) of user 110; a drive module 120 (e.g., drive modules 35 and 45 of fig. 3) configured to generate torque to be applied to the legs of user 110; and a control module 130 (e.g., control module 510 of fig. 5a and 5 b) configured to control the wearable device 100.
The wearable device 100 may include an angle sensor 125 configured to measure a joint angle of the user and an IMU135 configured to measure changes in acceleration and rotational speed from body movements of the user 110. The angle sensor 125 may measure a rotation angle (or angular velocity) of the leg support frame of the wearable device 100 corresponding to the hip angle value of the user 110. The rotation angle of the leg support frame measured by the angle sensor 125 may be estimated as a hip joint angle value (or leg angle value) of the user 110. The angle sensor 125 may include, for example, an encoder and/or a hall sensor. In an embodiment, the angle sensor 125 may be present near each of the right and left hip joints of the user 110. The IMU135 may include an acceleration sensor and/or an angular velocity sensor (e.g., a gyroscope sensor), and may measure changes in acceleration and/or angular velocity (or rotational velocity) from the motion of the user 110. The IMU135 may measure, for example, a upper body motion value of the user 110, where the upper body motion value corresponds to a motion value of a lumbar support frame (or base (base 80 of fig. 3)) of the wearable device 100. The motion value of the lumbar support frame measured by the IMU135 may be estimated as the upper body motion value of the user 110. Herein, the "IMU" may also be referred to as an "inertial sensor".
In an embodiment, the control module 130 and IMU135 may be disposed within a base (e.g., base 80 of fig. 3) of the wearable device 100. The substrate may be on the waist (or waist region) of the user 110 when the user 110 wears the wearable device 100. The matrix may be formed on the exterior of the lumbar support frame of the wearable device 100 or attached to the exterior of the lumbar support frame of the wearable device 100. The base may be mounted on the lumbar region of the user 110 to provide a cushioning feel to the lower back of the user 110 and may support the lower back of the user 110 along with a lumbar support frame.
Fig. 2 is a diagram illustrating a management system including a wearable device and an electronic device according to an embodiment.
Referring to fig. 2, exercise management system 200 may include a wearable device 100 to be worn on the body of a user, an electronic device 210, another wearable device 220, and a server 230. In embodiments, at least one of these devices (e.g., another wearable device 220 or server 230) may be omitted from exercise management system 200, or one or more other devices (e.g., a dedicated controller device of wearable device 100) may be added to exercise management system 200.
In embodiments, the wearable device 100 worn on the user's body may assist the user's motion in a walking assist mode. For example, the wearable device 100 worn on the user's leg may assist the user in walking by generating an assist force that assists the movement of the user's leg.
In an embodiment, in order to enhance an exercise effect on a user in the exercise assisting mode, the wearable device 100 may generate a resistance against the physical movement of the user or an assisting force assisting the physical movement of the user, and may apply such force to the body of the user. In the exercise assisting mode, through the electronic device 210, the user may select an exercise program (e.g., freehand squat, split leg squat, dumbbell squat, bowing, stretching, etc.) and/or an exercise intensity to be applied to the wearable device 100 that the user desires to exercise by using the wearable device 100. The wearable device 100 may control a driving module of the wearable device 100 according to an exercise program selected by a user, and may obtain sensor data including motion information of the user through a sensor module. The wearable device 100 may adjust the intensity of the resistance or the assisting force applied to the user according to the exercise intensity selected by the user. For example, the wearable device 100 may control the drive module to generate a resistance corresponding to the exercise intensity selected by the user.
In an embodiment, the wearable device 100 may be used to measure the physical fitness of a user by interacting with the electronic device 210. The wearable device 100 may operate in a physical ability measurement mode, which is a mode for measuring physical ability of a user, through control of the electronic device 210, and may transmit sensor data obtained through movement of the user in the physical ability measurement mode to the electronic device 210. The electronic device 210 may estimate the user's physical ability by analyzing the sensor data received from the wearable device 100.
The electronic device 210 may communicate with the wearable device 100 and may remotely control the wearable device 100 or provide status information to a user regarding the current status (e.g., an activated state, a charged state, a sensed state, or an error state) of the wearable device 100. The electronic device 210 may receive sensor data obtained by sensors in the wearable device 100 from the wearable device 100 and may estimate the user's physical ability or exercise results based on the received sensor data. In an embodiment, when a user exercises while wearing the wearable device 100, the wearable device 100 may obtain sensor data including motion information of the user using the sensor, and may transmit the obtained sensor data to the electronic device 210. The electronic device 210 may extract a motion value of the user from the sensor data and evaluate an exercise posture of the user based on the extracted motion value. The electronic device 210 may provide the user with exercise posture measurement values and exercise posture assessment information related to the user's exercise posture through a Graphical User Interface (GUI).
In an embodiment, the electronic device 210 may execute a program (e.g., an application) configured to control the wearable device 100, and the user may adjust the operation or set point of the wearable device 100 (e.g., the magnitude of the torque output from the driving module (e.g., the driving modules 35 and 45 of fig. 3), the volume of the audio output from the sound output module (e.g., the sound output module 550 of fig. 5a and 5 b), or the brightness of the lighting unit (e.g., the lighting unit 85 of fig. 3) through the corresponding program. The program executed by the electronic device 210 may provide a GUI for interaction with a user. The electronic device 210 may be a variety of forms of devices. For example, the electronic device 210 may include a portable communication device (e.g., a smart phone), a computer device, an access point, a portable multimedia device, or a home appliance (e.g., a television, an audio device, or a projection device), but examples are not limited to the foregoing devices.
In an embodiment, the electronic device 210 may connect to the server 230 using short range wireless communication or cellular communication. The server 230 may receive user profile information of a user of the wearable device 100 from the electronic device 210 and store and manage the received user profile information of the user. For example, the user profile information may include at least one of a name, an age, a gender, a height, a weight, and a Body Mass Index (BMI). Server 230 may receive exercise history information from electronic device 210 regarding exercises performed by the user, and may store and manage the received exercise history information. Server 230 may provide various exercise programs or fitness measurement programs to electronic device 210 to be provided to the user.
In an embodiment, the wearable device 100 and/or the electronic device 210 may be connected to another wearable device 220. Another wearable device 220 may include, for example, a wireless headset 222, a smart watch 224, or smart glasses 226, but examples are not limited to the foregoing devices. In an embodiment, the smart watch 224 may measure a bio-signal including heart rate information of the user and may send the measured bio-signal to the electronic device 210 and/or the wearable device 100. The electronic device 210 may estimate heart rate information (e.g., current heart rate, maximum heart rate, or average heart rate) of the user based on the bio-signal received from the smart watch 224, and may provide the estimated heart rate information to the user.
In an embodiment, the exercise result information, the physical ability information (e.g., gait evaluation information), and/or the exercise posture evaluation information of the user evaluated by the electronic device 210 may be transmitted to another wearable device 220, and may be provided to the user through the other wearable device 220. The status information of the wearable device 100 may be sent to another wearable device 220 and may be provided to the user by the other wearable device 220. In an embodiment, the wearable device 100, the electronic device 210, and the other wearable device 220 may be connected to each other by wireless communication (e.g., bluetooth TM or Wi-Fi communication).
In an embodiment, the wearable device 100 may provide (or output) feedback (e.g., visual, audible, or tactile feedback) corresponding to the state of the wearable device 100 according to the control signal received from the electronic device 210. For example, the wearable device 100 may provide visual feedback through a lighting unit (e.g., lighting unit 85 of fig. 3) and may provide audible feedback through a sound output module (e.g., sound output module 550 of fig. 5a and 5 b). The wearable device 100 may include a haptic module and provide haptic feedback to the user's body in the form of vibrations through the haptic module. The electronic device 210 may also provide (or output) feedback (e.g., visual feedback, auditory feedback, or tactile feedback) corresponding to the state of the wearable device 100.
In an embodiment, the electronic device 210 may present the personalized exercise target to the user in the exercise assistance mode. The personalized workout goals may include individual target workouts of a user's desired workout types (e.g., strength workouts, balance workouts, and aerobic workouts) as determined by the electronic device 210 and/or the server 230. When the server 230 determines the target exercise amount, the server 230 may transmit information about the determined target exercise amount to the electronic device 210. The electronic device 210 may personalize and present a target amount of exercise for the type of exercise (e.g., strength exercise, aerobic exercise, and balance exercise) according to the user's desired exercise program (e.g., freehand squat, split leg squat, or bowing knee) and/or physical characteristics (e.g., age, height, weight, and BMI). The electronics 210 can display a GUI screen on the display that displays the target workouts for each workout type.
In an embodiment, electronic device 210 and/or server 230 may include a database in which information regarding a plurality of exercise programs to be provided to a user by wearable device 100 is stored. To achieve the user's exercise goals, electronic device 210 and/or server 230 may recommend an exercise program appropriate for the user. The exercise goal may include, for example, at least one of improved muscle strength, improved physical strength, improved cardiovascular endurance, improved core stability, improved flexibility, or improved symmetry. Electronic device 210 and/or server 230 may store and manage exercise programs performed by a user, results of performing exercise programs, and the like.
According to an embodiment, the electronic device 210 may evaluate the user's walking ability by interacting with the wearable device 100. For example, the electronic device 210 may estimate a gait index based on sensor data obtained from a sensor module of the wearable device 100 worn by the user, wherein the gait index is an indicator of the user's walking state. The gait index may be a criterion for determining the quality of walking performed by the user. The gait index estimated by the electronic device 210 may include, for example, at least one of a walking speed, a step time, a step size of one step, a step length of two steps, a walking distance, a gait symmetry index, a gait change index or a walking ratio. The electronic device 210 may calculate the gait index in real-time as the user walks wearing the wearable device 100. The stride length and stride length are described in detail with reference to FIG. 10.
When a user wears the wearable device 100 to walk (or exercise), the wearable device 100 may obtain sensor data including motion information related to the user's walking using the sensor, and may transmit the obtained sensor data to the electronic device 210. The electronics 210 may estimate gait evaluation information of the user based on the sensor data and may provide the estimated gait evaluation information to the user. Gait assessment information may include, for example, various gait indices related to the user's walking (e.g., walking speed, step time, stride length, gait symmetry index, gait change index, and walking ratio) when the user walks while wearing the wearable device 100. The electronic device 210 may provide feedback information to the user based on the gait evaluation information to improve the user's walking state. For example, when the user's measured step size/stride length is determined to be less than the desired step size/stride length, the electronic device 210 may suggest that the user walk at a wider step size/stride length.
The wearable device 100 and the electronic device 210 may estimate a gait index, such as a walking speed, by using an angle sensor (e.g., angle sensor 125) and an IMU (e.g., IMU 135) of the wearable device 100, without using a Global Positioning System (GPS) sensor to track a user's position, and thus, the gait index may be estimated not only outdoors but also indoors. In addition, even when the user walks at a fixed location, such as on a treadmill, the wearable device 100 and the electronic device 210 may estimate the gait index and may estimate the gait index reflecting the individual characteristics of the user. The electronic device 210 may provide the user using the wearable apparatus 100 with evaluation information on the walking state of the user to increase the user's interest in walking or exercise.
Fig. 3 is a rear schematic view illustrating a wearable device according to an embodiment. Fig. 4 is a left side view illustrating a wearable device according to an embodiment.
Referring to fig. 3 and 4, the wearable device 100 according to the embodiment may include a base 80, a lumbar support frame 20, driving modules 35 and 45, leg support frames 50 and 55, thigh fastening portions 1 and 2, and a lumbar fastening portion 60. The base 80 may include a lighting unit 85. In embodiments, at least one of these components (e.g., lighting unit 85) may be omitted from wearable device 100, or one or more other components (e.g., haptic modules) may be added to wearable device 100.
The substrate 80 may be on the waist of the user when the user wears the wearable device 100. The base 80 worn on the waist of the user may cushion and support the waist of the user. When the user wears the wearable device 100, the base 80 may be above the user's hip so that the wearable device 100 does not deflect downward due to gravity. The base 80 may distribute some of the weight of the wearable device 100 to the waist of the user when the user wears the wearable device 100. The base 80 may be connected to the lumbar support frame 20. Lumbar support frame connecting elements (not shown) that may be connected to the lumbar support frame 20 may be located at both edges of the base 80.
In an embodiment, the lighting unit 85 may be disposed outside the base 80. The lighting unit 85 may include a light source (e.g., a Light Emitting Diode (LED)). The lighting unit 85 may emit light by control of a control module (not shown), for example, the control module 510 of fig. 5a and 5 b. In some embodiments, the control module may control the lighting unit 85 to provide (or output) visual feedback to the user corresponding to the state of the wearable device 100 through the lighting unit 85.
The lumbar support frame 20 may extend from both edges of the base 80. The user's lumbar may be housed inside the lumbar support frame 20. The lumbar support frame 20 may include one or more rigid beams. Each beam may be a curved shape having a preset curvature such that the beam may encircle the waist of the user. The lumbar fasteners 60 may be attached to the edges of the lumbar support frame 20. The drive modules 35 and 45 may be connected to the lumbar support frame 20.
In an embodiment, a control module, an IMU (not shown) (e.g., IMU135 of fig. 1 or IMU522 of fig. 5 b), a communication module (not shown) (e.g., communication module 516 of fig. 5a and 5 b), and a battery (not shown) may be disposed within the matrix 80. The base 80 may protect the control module, IMU, communication module, and battery. The control module may generate control signals for controlling the operation of the wearable device 100. The control module may include control circuitry including a processor and memory configured to control the actuators of the drive modules 35 and 45. The control module may also include a power module (not shown) to provide power from the battery to each component in the wearable device 100.
In an embodiment, the wearable device 100 may include a sensor module (not shown) (e.g., sensor module 520 of fig. 5 a) for obtaining sensor data from one or more sensors. The sensor module may obtain sensor data that varies according to the movement of the user. In an embodiment, the sensor module may obtain sensor data including motion information of a user and/or motion information of a component of the wearable device 100. The sensor module may include, for example, but is not limited to, an IMU (e.g., IMU135 of fig. 1 or IMU522 of fig. 5 b) configured to measure a user's upper body motion value or a lumbar support frame 20 motion value and an angle sensor (e.g., angle sensor 125 of fig. 1 or first and second angle sensors 520-1 of fig. 5 b) configured to measure a user's hip joint angle value or a leg support frame 50 and 55 motion value. For example, the sensor module may further include at least one of a position sensor, a temperature sensor, a bio-signal sensor, and a proximity sensor.
The lumbar fasteners 60 may be connected to the lumbar support frame 20 and secure the lumbar support frame 20 to the user's lumbar. The waist fastener 60 may comprise, for example, a waistband.
The driving modules 35 and 45 may generate an external force (or torque) to be applied to the body of the user based on the control signal generated by the control module. For example, the drive modules 35 and 45 may generate an assist force or resistance to be applied to the legs of the user. In an embodiment, the driving modules 35 and 45 may include a first driving module 45 and a second driving module 35, wherein the first driving module 45 is at a position corresponding to a position of a right hip joint of the user and the second driving module 35 is at a position corresponding to a position of a left hip joint of the user. The first drive module 45 may include a first actuator and a first joint member, and the second drive module 35 may include a second actuator and a second joint member. The first actuator may provide a force to be transferred to the first joint member and the second actuator may provide a force to be transferred to the second joint member. The first and second actuators may each include an engine configured to generate a force (or torque) by receiving power from a battery. When driven by receiving electric power, the engine may generate a force (assist force) for assisting the body movement of the user or a force (resistance force) for obstructing the body movement of the user. In an embodiment, the control module may adjust the strength and direction of the force generated by the motor by adjusting the voltage and/or current supplied to the motor.
In an embodiment, the first and second joint members may receive forces from the first and second actuators, respectively, and may apply an external force to the user's body based on the received forces. The first and second joint members may be respectively in positions corresponding to joints of a user. One side of the first joint member may be connected to the first actuator and the other side of the first joint member may be connected to the first leg support frame 55. The first joint member is rotatable by a force transmitted from the first actuator. An encoder or a hall sensor may be arranged on one side of the first joint member, wherein the encoder or hall sensor may operate as an angle sensor configured to measure a rotation angle of the first joint member (corresponding to a joint angle of a user). One side of the second joint member may be connected to the second actuator and the other side of the second joint member may be connected to the second leg support frame 50. The second joint member is rotatable by a force transmitted from the second actuator. An encoder or a hall sensor may be arranged on one side of the second joint member, wherein the encoder or hall sensor may operate as an angle sensor configured to measure the rotation angle of the second joint member.
In an embodiment, the first actuator may be in a lateral direction of the first joint member and the second actuator may be in a lateral direction of the second joint member. The rotation axis of the first actuator and the rotation axis of the first joint member may be spaced apart from each other, and the rotation axis of the second actuator and the rotation axis of the second joint member may be spaced apart from each other. However, the examples are not limited to the foregoing examples, and the actuator and the joint member may share a rotation axis. In an embodiment, the actuators may be spaced apart from the joint members, respectively. In this case, the driving modules 35 and 45 may further include a force transmission module (not shown) for transmitting force from the actuator to the joint member. The force transmission module may be a rotating body, such as a gear, or a longitudinal member, such as a wire, cable, rope, spring, belt or chain. However, the scope of the embodiments is not limited by the force transmission structure described above and the positional relationship between the actuator and the joint member.
In an embodiment, the leg support frames 50 and 55 may support a user's legs (e.g., thighs) when the wearable device 100 is worn on the user's legs. For example, the leg support frames 50 and 55 may transmit the force (torque) generated by the driving modules 35 and 45 to the thighs of the user, and the force may be used as an external force to be applied to the movement of the legs of the user. When the edge of each of the leg support frames 50 and 55 is connected to the joint member to rotate and the other edge of each of the leg support frames 50 and 55 is connected to the thigh fastening parts 1 and 2, the leg support frames 50 and 55 may transmit the force generated by the driving modules 35 and 45 to the user's thighs while supporting the user's thighs. For example, the leg support frames 50 and 55 may push or pull the thighs of the user. The leg support frames 50 and 55 may extend in the longitudinal direction of the user's thighs. Leg support frames 50 and 55 are bendable to respectively encircle at least a portion of the thigh circumference of the user. The leg support frames 50 and 55 may include a first leg support frame 55 for supporting the user's right leg and a second leg support frame 50 for supporting the user's left leg.
The thigh fastening portions 1 and 2 may be connected to the leg support frames 50 and 55, and may fix the leg support frames 50 and 55 to the thighs, respectively. The thigh fastening parts 1 and 2 may include a first thigh fastening part 2 for fixing the first leg supporting frame 55 to the user's right leg and a second thigh fastening part 1 for fixing the second leg supporting frame 50 to the user's left leg.
In an embodiment, the first thigh fastening part 2 may comprise a first cover, a first fastening frame and a first strap, and the second thigh fastening part 1 may comprise a second cover, a second fastening frame and a second strap. The first and second covers may apply torque generated by the driving modules 35 and 45 to the thighs of the user. The first cover and the second cover may be positioned at one side of the user's thigh to push or pull the user's thigh. The first and second covers may be on the front surface of the user's thigh. The first cover and the second cover may be in a circumferential direction of the user's thigh. The first and second covers may extend from the other edges of the leg support frames 50 and 55 to both sides, and may include curved surfaces corresponding to thighs of the user. Edges of the first and second covers may be connected to the fastening frame, and the other edges of the first and second covers may be connected to the strap.
The first and second fastening frames may be arranged to, for example, enclose at least a portion of the thigh circumference of the user and prevent separation of the thigh of the user from the leg support frames 50 and 55. The first fastening frame may have a fastening structure connecting the first cover to the first strap, and the second fastening frame may have a fastening structure connecting the second cover to the second strap.
The first strap may encircle a remaining portion of the user's right leg band that is not encircled by the first cover and the first fastening frame, and the second strap may encircle a remaining portion of the user's left leg band that is not encircled by the second cover and the second fastening frame. The first and second belts may comprise, for example, an elastic material (e.g., a belt).
Fig. 5a and 5b are diagrams each showing a configuration of a control system of a wearable device according to an embodiment.
Referring to fig. 5a, a wearable device (e.g., the wearable device 100 of fig. 1 and 3) may be controlled by a control system 500. The control system 500 may include a control module 510, a sensor module 520, a drive module 530, and a battery 540. The drive module 530 may include a motor 534 for generating a force (e.g., torque) and a motor drive circuit 532 for driving the motor 534. Although one sensor module 510 and a drive module 530 including one engine drive circuit 532 and one engine 534 are shown in the example of fig. 5a, this is only one example. Referring to fig. 5b, as shown in the illustrated example, a plurality (e.g., two or more) of sensor modules 520 and 520-1, engine drive circuits 532 and 532-1, and engines 534 and 534-1 may be provided. The drive module 530 including the engine drive circuit 532 and the engine 534 may correspond to the first drive module 45 of fig. 3, and the drive module 530-1 including the engine drive circuit 532-1 and the engine 534-1 may correspond to the second drive module 35 of fig. 3. The following description of each of the sensor module 520, the engine drive circuit 532, and the engine 534 is also applicable to the sensor module 520-1, the engine drive circuit 532-1, and the engine 534-1 shown in fig. 5 b.
Referring to fig. 5a, the sensor module 520 may include at least one sensor. The sensor module 520 may include sensor data, wherein the sensor data includes motion information of a user. For example, the sensor module 520 may obtain sensor data that varies according to the user's leg movements. The sensor module 520 may send the obtained sensor data to the control module 510. The sensor module 520 may include, for example, an IMU, an angle sensor (e.g., encoder), a position sensor, a proximity sensor, a bio-signal sensor, or a temperature sensor. The IMU may measure acceleration information and posture information as the user walks. For example, the IMU may sense acceleration of each of the X, Y, and Z axes and angular velocity of each of the X, Y, and Z axes according to user gait motion. The angle sensor may measure angle information about the user's hip joint angle. The angle information measured by the angle sensor may include, for example, a hip joint angle of the right leg, a hip joint angle of the left leg, and information about a movement direction of the leg.
The battery 540 may power each component of the wearable device. The wearable device may convert the power of the battery 540 into power suitable for the operation voltage of each component in the wearable device and supply the converted power to each component.
The driving module 530 may generate an external force to be applied to the user's leg according to the control of the control module 510. The drive module 530 may be in a position corresponding to the position of the user's hip joint and may generate torque to be applied to the user's leg based on control signals generated by the control module 510. The control module 510 may send a control signal to the motor drive circuit 532, and the motor drive circuit 532 may control operation of the motor 534 by generating a current signal (or voltage signal) corresponding to the control signal and supplying the current signal to the motor 534. According to the control signal, the current signal may not be supplied to the motor 534. When a current signal is supplied to the motor 534 and the motor is driven, the motor 534 may generate a force for assisting the movement of the user's legs or a torque for obstructing the movement of the user's legs.
The control module 510 may control the overall operation of the wearable device and may generate control signals that control each component (e.g., the drive module 530). The control module 510 may include a processor 512, a memory 514, and a communication module 516.
The processor 512 may run, for example, software to control at least one other component (e.g., hardware or software component) of the wearable device connected to the processor 512, and may perform various types of data processing or operations. In an embodiment, the processor 512 may store instructions or data received from another component (e.g., the communication module 516) in the memory 514, may process the instructions or data stored in the memory 514, and may store the resulting data after processing in the memory 514 as at least part of the data processing or operation. In an embodiment, the processor 512 may include a main processor (e.g., a Central Processing Unit (CPU) or an Application Processor (AP)) or an auxiliary processor (e.g., a Graphics Processor (GPU), a Neural Processor (NPU), an Image Signal Processor (ISP), a sensor hub processor, or a Communication Processor (CP)) that operates independently of or with the main processor. The secondary processor may be implemented separately from the primary processor or as part of the primary processor.
The memory 514 may store various data used by at least one component of the control module 510 (e.g., the processor 512). The various data may include, for example, sensor data, software, input data for instructions associated therewith, or output data. Memory 514 may include volatile memory or nonvolatile memory such as Random Access Memory (RAM), dynamic RAM (DRAM), or Static RAM (SRAM).
The communication module 516 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the control module 510 and another component of the wearable device or an external electronic device (e.g., the electronic device 210 or another wearable device 220 of fig. 2) and performing communication via the established communication channel. For example, the communication module 516 may transmit the user's sensor data and/or gait assessment information to an external electronic device. According to an embodiment, the communication module 516 may include one or more CPs that are operable independent of the processor 512 and support direct (e.g., wired) or wireless communication. In an embodiment, the communication module 516 may include a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a Global Navigation Satellite System (GNSS) communication module) and/or a wired communication module. A respective one of these communication modules may communicate with other components of the wearable device and/or external electronic devices via a short-range communication network, such as bluetooth TM, wi-Fi, ANT, or infrared data association (IrDA), or a long-range communication network, such as a conventional cellular network, 5G network, next-generation communication network, the internet, or a computer network, such as a Local Area Network (LAN) or Wide Area Network (WAN).
Fig. 6 is a diagram illustrating interactions between a wearable device and an electronic device according to an embodiment.
Referring to fig. 6, the wearable device 100 may communicate with an electronic device 210. For example, the electronic device 210 may be a user terminal of a user using the wearable device 100 or a controller device dedicated to the wearable device 100. In an embodiment, the wearable device 100 and the electronic device 210 may be connected to each other by short-range wireless communication (e.g., bluetooth TM or Wi-Fi communication).
In an embodiment, the electronic device 210 may check the status of the wearable device 100 or may execute an application to control or operate the wearable device 100. A screen of a User Interface (UI) may be displayed on the display 212 of the electronic device 210 by executing an application program to control the operation of the wearable device 100 or to determine an operation mode of the wearable device 100. For example, the UI may be a GUI.
In an embodiment, a user may input commands (e.g., commands to perform a walking assist mode, an exercise assist mode, or a fitness measurement mode) through a GUI screen on the display 212 of the electronic device 210 to control the operation of the wearable device 100, or may change settings of the wearable device 100. The electronic device 210 may generate a control command (or control signal) corresponding to an operation control command or a setting change command input by a user, and may transmit the generated control command to the wearable device 100. The wearable device 100 may operate according to the received control command, and may transmit a result of the execution according to the received control command and/or sensor data sensed by a sensor module of the wearable device 100 to the electronic device 210. The electronic device 210 may provide the user with result information (e.g., gait ability information, exercise ability information, or exercise movement assessment information) obtained by analyzing the control result and/or sensor data through the GUI screen.
Fig. 7 is a diagram showing an electronic device configuration according to an embodiment.
Referring to fig. 7, the electronic device 210 may include a processor 710, a memory 720, a communication module 730, a display module 740, a sound output module 750, and an input module 760. In embodiments, at least one of these components (e.g., the sound output module 750) may be omitted from the electronic device 210, or one or more other components (e.g., the sensor module and the battery) may be added thereto.
The processor 710 may control at least one other component (e.g., hardware or software component) of the electronic device 210 and may perform various types of data processing or operations. In an embodiment, processor 710 may store instructions or data received from another component (e.g., communication module 730) in memory 720, may process instructions or data stored in memory 720, and may store resulting data in memory 720, at least as part of the data processing or operation.
In embodiments, processor 710 may include a main processor (e.g., a CPU or AP) or may be a secondary processor (e.g., GPU, NPU, ISP, a sensor hub processor, or a CP) that operates independently of or in conjunction with the main processor.
Memory 720 may store various data used by at least one component of electronic device 210 (e.g., processor 710 or communication module 730). The various data may include, for example, programs (e.g., applications) and input data or output data for commands associated therewith. Memory 720 may include at least one instruction that may be executed by processor 710. Memory 720 may include volatile memory or nonvolatile memory.
The communication module 730 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 210 and another electronic device (e.g., the wearable device 100, the other wearable device 220, and the server 230) and performing communication through the established communication channel. The communication module 730 may include communication circuitry for performing communication functions. The communication module 730 may include one or more CPs that may operate independently of the processor 710 (e.g., an AP) and support direct (e.g., wired) or wireless communication. In an embodiment, the communication module 730 may include a wireless communication module (e.g., a bluetooth TM communication module, a cellular communication module, a Wi-Fi communication module, or a GNSS communication module) or a wired communication module (e.g., a LAN communication module or a Power Line Communication (PLC) module) that performs wireless communication. For example, the communication module 730 may transmit a control command to the wearable device 100, and may receive at least one of sensor data including body movement information of a user wearing the wearable device 100, state data of the wearable device 100, and control result data corresponding to the control command from the wearable device 100.
The display module 740 may visually provide information to the outside (e.g., user) of the electronic device 210. The display module 740 may include, for example, a Light Emitting Diode (LED) or Organic Light Emitting Diode (OLED) display, a holographic device, or a projection device. The display module 740 may also include control circuitry for controlling the driving of the display. In an embodiment, the display module 740 may also include a touch sensor configured to sense a touch or a pressure sensor configured to sense the intensity of force generated by a touch.
The sound output module 750 may output the sound signal to the outside of the electronic device 210. The sound output module 750 may include a guide sound signal (e.g., a driving start sound or an operation error notification sound) based on the state of the wearable device 100 and a speaker for playing music content or guide voice. When it is determined that the wearable device 100 is not properly worn on the user's body, the sound output module 750 may output a guiding voice to inform the user that they are abnormally wearing the wearable device 100 or guiding the user to normally wear the wearable device 100. The sound output module 750 may output, for example, guide voice corresponding to exercise evaluation information or exercise result information obtained by evaluating exercise of the user.
The input module 760 may receive commands or data from outside the electronic device 210 (e.g., a user) to be used by another component of the electronic device 210 (e.g., the processor 710). The input module 760 may include input component circuitry and receive user input. The input module 760 may include, for example, keys (e.g., buttons) or a touch screen.
According to an embodiment, the electronic device 210 may include a communication module 730, a processor 710, and a display module 740, wherein the communication module 730 is configured to receive sensor data from the wearable device 100 including movement information of a user wearing the wearable device 100, the processor 710 is configured to estimate a gait index indicating a walking state of the user based on the sensor data, and the display module 740 is configured to output the physical ability information including the gait index.
In an embodiment, the processor 710 may estimate a gait index indicative of the user's walking state based on sensor data received from the wearable device 100 (e.g., sensor data obtained by an IMU and/or an angle sensor of the wearable device 100). Processor 710 may determine an average walking speed over a particular time interval based on the estimated walking speed. The processor 710 may estimate the walking speed of the user by using a walking speed estimation model having sensor data as input. Processor 710 may estimate the user's walking speed in real-time through a walking speed estimation model. In an embodiment, angular velocity (rotational velocity) data (e.g., angular velocity data of each of the x-axis, y-axis, and z-axis) and acceleration data (e.g., acceleration data of each of the x-axis, y-axis, and z-axis) obtained by the IMU of the wearable device 100 may be input to the walking speed estimation model, and the walking speed estimation model may output a walking speed estimation value based on the input angular velocity data and the input acceleration data. In an embodiment, hip joint angular velocity data (e.g., angular velocity data corresponding to a right hip joint and angular velocity data corresponding to a left hip joint) obtained by an angle sensor of the wearable device 100 and angular velocity (rotational velocity) data and acceleration data obtained by an IMU of the wearable device 100 may be input to a walking speed estimation model, and the walking speed estimation model may output a walking speed estimation value based on the input angular velocity data and acceleration data of the IMU and the input angular velocity data of the angle sensor. In an embodiment, rotation angle data (e.g., joint angle of right hip joint and joint angle of left hip joint) and hip joint angular velocity data obtained by an angle sensor of the wearable device 100 may be input to the walking speed estimation model together with angular velocity (rotation speed) data and acceleration data obtained by an IMU of the wearable device 100, and the walking speed estimation model may output a walking speed estimation value based on the input angular velocity data and acceleration data of the IMU and the input rotation angle data and angular velocity data of the angle sensor. In an embodiment, rotation angle data (e.g., joint angle of right hip joint and joint angle of left hip joint) and hip joint angular velocity data obtained by the angle sensor of the wearable device 100 may be input to the walking speed estimation model, and the walking speed estimation model may output a walking speed estimation value based on the input rotation angle data and angular velocity data of the angle sensor.
The walking speed estimation model may be a model trained to input and output a walking speed estimation value corresponding to the input sensor data (e.g., sensor data obtained by the IMU and/or sensor data obtained by the angle sensor). For example, the walking speed estimation model may be a model trained by machine learning or linear regression based on training data (e.g., sensor data measured during walking and actual walking speed values corresponding to the sensor data). In embodiments, the walking speed estimation model may be, for example, a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Gated Recurrent Unit (GRU), resNet, long-term short-term memory (LSTM), or a linear regression model, or any combination of two or more thereof (e.g., a combination of CNN and LSTM). The walking speed estimation model may be trained based on a supervised learning machine learning approach. During the training process, an error back-propagation algorithm or a gradient descent algorithm may be used. For example, in the training process, determination and adjustment may be repeatedly performed, wherein the determination of the loss is based on a difference between an actual walking speed value and a walking speed estimation value output by a walking speed estimation model to which sensor data is input, and adjusting a parameter (e.g., a connection weight or bias) of the walking speed estimation model is performed to reduce the loss. Through this training process, the optimum value of the hyper-parameter in the form of the walking speed estimation model can be explored and determined. The training of the walking speed estimation model is described below with reference to fig. 13 and 14.
In an embodiment, processor 710 may estimate a step time of the user based on the sensor data. The step time may be the time it takes for the user to walk one step, and the processor 710 may measure the step time of a single step when the user walks with the wearable device worn. For example, the processor 710 may detect a start of a step of the user and an end of the step of the user based on the sensor data, and may estimate a step time during the step of the user based on the detected start and end of the step. In an embodiment, the processor 710 may measure a time interval between points at which the left and right hip angles measured by the angle sensor of the wearable device 100 intersect each other or a time interval between points at which the angular velocity of the left and right hip joints calculated from the left and right hip angles intersect each other, and may determine the measured time interval as a step time of the user. In another embodiment, the processor 710 may identify a point in time when the user's foot contacts the ground and a point in time when the user's foot is off the ground based on sensor data measured by the IMU of the wearable device 100, and may determine a time interval between the identified points in time as the user's step time. In yet another embodiment, to estimate the step time, processor 710 may use a step time estimation model trained using sensor data regarding hip angle (or sensor data measured by the IMU) during walking and point-in-time data regarding the actual point in time when the user's foot contacts the ground and the user's foot is off the ground as training data. The step time estimation model may be a neural network model trained according to the machine learning process or linear regression algorithm described above.
In embodiments, the processor 710 may estimate one or more other gait indices based on the walking speed and the step time. In this case, the other gait index means a gait index other than the above-described walking speed and step time. The other gait index may include, for example, at least one of a step size of one step, a step length of two steps, a walking distance, a gait symmetry index, a gait change index and a walking ratio. The processor 710 may estimate other gait indices in real time.
In an embodiment, processor 710 may determine a step size/stride length based on the sensor data. Processor 710 may estimate an average walking speed during a step of the user based on the sensor data and determine a step size of the step based on the average walking speed and the step size time of the step. Processor 710 may determine a step size of a step by multiplying the average walking speed by a step time during the step. In an embodiment, processor 710 may estimate a step size of a step by using a model trained by a machine learning process or a linear regression algorithm based on training data (e.g., sensor data and a step size of an actual step).
In an embodiment, processor 710 may estimate a left stride length of the user and a right stride length of the user based on a first hip angle value when the hip angle of the right leg of the user is at a maximum, a second hip angle value when the hip angle of the left leg of the user is at a maximum, and a two-step stride length of the user.
In an embodiment, the processor 710 may determine a gait symmetry index based on the sensor data. The gait symmetry index may be a value indicating the degree of symmetry of the gait of the user's right leg (corresponding to the right step) and the gait of the user's left leg (corresponding to the left step). Processor 710 may determine an average of the user's left step size and an average of the user's right step size based on the sensor data. Processor 710 may determine a gait symmetry index for the user's gait motion based on the average of the left step size and the average of the right step size. The processor 710 may determine a left step size and a right step size based on the estimation of the step sizes, and may determine a gait symmetry index based on a difference between an average of the left step size and an average of the right step size. When the difference is large, the gait symmetry index may decrease, and when the difference is small, the gait symmetry index may increase, but the example is not limited thereto. In another embodiment, processor 710 may determine an average step time for the left step of the user and an average step time for the right step of the user based on the sensor data. Processor 710 may determine a gait symmetry index for the user's gait motion based on the average step time for the left step and the average step time for the right step. Processor 710 may determine an average step time for the left step and an average step time for the right step based on the above-described estimation of step times, and may determine a gait symmetry index based on a difference between the average step time for the left step and the average step time for the right step.
In an embodiment, the processor 710 may determine a gait change index based on the sensor data. The gait change index may be an index indicating the regularity or variability of the user's repeated gait movements. The processor 710 may determine a gait change index based on a variability index such as a standard deviation of a step size or a standard deviation of a stride time of two steps. For example, the processor 710 may determine a gait change index of the user's gait motion during the user's gait motion based on a standard deviation of the stride length of two steps measured during the predefined number of steps or a standard deviation of the stride time of two steps measured during the predefined number of steps. When the standard deviation is large, the gait change index may increase, and when the standard deviation is small, the gait change index may decrease, but the example is not limited thereto.
In an embodiment, the processor 710 may determine the walking ratio by dividing the average of the steps of one step of the user by the number of steps per minute. The number of steps per minute may be determined based on an average of the step length times of one step. In another embodiment, the processor 710 may determine the walking ratio by dividing the average of the user's walking speed by the square of the number of steps per minute. The walking ratio may be used as an indicator to estimate the likelihood of a user falling. In yet another embodiment, for the estimation of the walking ratio, the processor 710 may use a walking ratio estimation model, wherein the walking ratio estimation model is trained using training data comprising sensor data (or sensor data measured by IMU and/or angle sensors) and information about the actual walking ratio. The walking ratio estimation model may be a neural network model trained according to the machine learning process or linear regression algorithm described above.
The processor 710 may determine a gait score corresponding to the gait evaluation result of the user's gait movement based on the gait index estimated as described above. In an embodiment, the processor 710 may determine the gait score based on the walking speed, the walking ratio, the gait symmetry index and the gait change index. For example, the processor 710 may determine an average, a sum, or a weighted sum of the gait speed, the walking ratio, the gait symmetry index, and the gait change index as a gait score, but the example is not limited thereto. The processor 710 may generate gait assessment information including a gait score and one or more gait indices. The generated gait evaluation information may be provided to the user through the display module 740 and/or the sound output module 750. According to an embodiment, gait evaluation information may be provided to the user by the wearable device 100.
Fig. 8 is a flowchart illustrating a method of providing a gait index of a user wearing a wearable device according to an embodiment. According to an embodiment, the method of providing a gait index may be performed by the electronic device 210.
Referring to fig. 8, in operation 810, the electronic device 210 may receive sensor data including motion information of a user wearing the wearable device 100 from the wearable device 100. The sensor data may include sensor data about acceleration and rotation speed according to body movement of the user and sensor data about joint angle (e.g., hip joint angle) of the user.
In operation 820, the electronic device 210 may estimate a gait index indicating a walking state of the user based on the sensor data. In an embodiment, the electronic device 210 may estimate the user's walking speed by using a walking speed estimation model having sensor data as input. The electronics 210 can estimate a user's step time based on the sensor data and can estimate other gait indices based on the walking speed and the step time. The electronics 210 can estimate other gait indices including, for example, at least one of a step size of one step, a step length of two steps, a walking distance, a gait symmetry index, a gait change index and a walking ratio. For example, the electronic device 210 may determine the walking ratio by dividing an average of steps of one step of the user by steps per minute or by dividing an average of walking speeds of the user by a square of steps per minute. The process of estimating each of the gait indexes is described in detail with reference to fig. 9.
In operation 830, the electronic device 210 may provide the estimated gait index to the user. Providing the gait index to the user may include an operation of determining a gait score corresponding to a gait evaluation result of the gait motion of the user based on the estimated gait index, and an operation of providing gait evaluation information including the determined gait score and the estimated gait index to the user. The electronic device 210 may provide the estimated gait index to the user via the display module and/or the sound output module. In an embodiment, the user may view the gait index through a program (e.g., application) installed in the electronic device 210.
The electronics 210 may determine a gait score corresponding to a gait evaluation result of the user's gait motion based on the estimated gait index and may provide gait evaluation information including the determined gait score and the estimated one or more gait indices to the user. The electronics 210 can determine the gait score based on the walking speed, the walking ratio, the gait symmetry index, and the gait change index. The walking speed is a representative gait index that generally decreases with age. Gait symmetry index and gait change index may be poor when there is a physical abnormality in the musculoskeletal or nervous system. Gait change index tends to decrease with age and is related to the likelihood of a fall. In embodiments, the gait score may be determined based on an average or weighted average of such gait indices (e.g., gait speed, gait symmetry index, gait change index and walking ratio), and the determined gait score may be provided to the user. In an embodiment, there may be a correlation in which the gait score increases as the walking ratio increases and the gait symmetry index and gait change index decrease.
Fig. 9 is a flowchart illustrating a method of estimating gait index according to an embodiment. According to an embodiment, the method of estimating gait index may be performed by the electronic device 210. In an embodiment, at least one of the operations of fig. 9 may be performed simultaneously or in parallel with each other, and the order of the operations may be changed. In addition, at least one of the operations may be omitted or another operation may be additionally performed.
Referring to fig. 9, in operation 910, the electronic device 210 may receive sensor data including motion information of a user wearing the wearable device 100 from the wearable device 100. During walking with the user wearing the wearable device 100, the IMU embedded in the wearable device 100 may measure acceleration and rotational speed around the user's pelvis in each of the up-down, front-back, and left-right directions, and may output the measured sensor data. The sensor data output from the IMU may include a value of acceleration on each of the three axes and a value of rotational speed on each of the three axes.
In operation 920, the electronic device 210 may determine whether the user's one step is ended based on the sensor data received from the wearable device 100. In an embodiment, the electronic device 210 may determine whether a step is ended by identifying a start and an end of the step based on the sensor data. The criteria for determining the start and end of a step may be arbitrarily determined. For example, the moment when the user's foot contacts the ground or the moment when the user's foot leaves the ground may be the determination criterion. For another example, the timing at which the left and right hip joint angle values intersect each other or the timing at which the left and right hip joint angular velocity values intersect each other may be a determination criterion. In an embodiment, the electronic device 210 may estimate the start and end of a step by using a model that is trained to distinguish between the start and end of a step when sensor data is entered.
When the user's step is not finished ("no" in operation 920), the electronic device 210 may estimate an instantaneous walking speed of the user based on the sensor data in operation 930. In an embodiment, sensor data obtained by the IMU and/or angle sensor of the wearable device 100 may be input to the above-described walking speed estimation model, and the electronic device 210 may obtain a walking speed estimation value corresponding to the user's instantaneous walking speed through the walking speed estimation model. The walking speed estimation model may continuously output the walking speed estimation value for a one-step time interval.
In operation 940, the electronic device 210 may update the average walking speed of one step of the user based on the instantaneous walking speed. For example, the electronic device 210 may update the average walking speed by averaging the accumulated instantaneous walking speed from the start of one step to the current point in time.
In operation 950, the electronic device 210 may record a hip angle value based on the angle sensor. In an embodiment, operation 950 may be optionally performed. When the hip angle value of the rear leg is maximum during one step, the electronic device 210 may record the hip angle value of the front leg and the hip angle value of the rear leg. The hip angle values thus recorded can be used to determine the step size of the user.
When the user's step is over ("yes" in operation 920), the electronic device 210 may determine a step length time of the user's step based on the sensor data in operation 960. For example, as described in operation 920, the electronic device 210 may detect a start of a step of the user and an end of the step of the user based on the sensor data, and may estimate a step time during the step of the user based on the detected start and end of the step.
In operation 970, the electronic device 210 may determine an average walking speed of the user. The electronic device 210 may perform the update process of the average walking speed in step 940 at one-step intervals, and may determine the average walking speed of the step as a result of the execution.
In operation 980, the electronic device 210 may determine a step size of the user. For example, the electronic device 210 may determine the step size of a step by multiplying the average walking speed during the step by the step size time of the step. For another example, the electronics 210 can determine the step size by using a model trained to estimate the step size based on the sensor data. For another example, the electronic device 210 may estimate the left step size of the user and the right step size of the user based on a first hip angle value when the hip angle of the right leg of the user is maximum, a second hip angle value when the hip angle of the left leg of the user is maximum, and a two-step stride length of the user. A method of estimating the left and right steps based on the hip angle values is described in detail below with reference to fig. 11a and 11 b.
In operation 990, the electronic device 210 may determine a gait symmetry index and a gait change index. The gait symmetry index may correspond to an index indicating a difference between a left step size and a right step size or a difference between a step size time of the left step and a step size time of the right step. For example, the electronics 210 can calculate an average of a left step size and an average of a right step size of a user's steps, and can determine a gait symmetry index based on a difference between the average of the left step size and the average of the right step size. For another example, the electronics 210 can determine the gait symmetry index based on a difference between a step time of a left step and a step time of a right step of a user's steps.
The electronics 210 may determine a gait change index of the user's gait motion during the user's gait motion based on a standard deviation of the stride length of two steps measured during a predefined number of steps (e.g., 10 or 20 steps) or a standard deviation of the stride time of two steps measured during the predefined number of steps. For example, the electronic device 210 may determine the gait change index as a result of dividing the standard deviation of the stride length of two steps by the average of the stride length of two steps, or as a result of dividing the standard deviation of the stride time of two steps by the average of the stride time of two steps. For another example, the electronics 210 can determine a standard deviation of a stride length of two steps or a standard deviation of a stride time of two steps as the gait change index. For another example, the electronic device 210 may determine a difference between a maximum value and a minimum value of the stride length of two steps or the stride time of two steps measured during the predefined number of steps or a difference between an upper limit value and a lower limit value of a certain percentage as the gait change index.
In an embodiment, the electronic device 210 may determine the walking ratio by dividing the average of the step sizes of one step of the user by the number of steps per minute, or by dividing the average of the walking speed of the user by the square of the number of steps per minute. In this case, the number of steps per minute may correspond to (average of 60/step length).
Fig. 10 is a diagram illustrating stride length according to an embodiment.
Fig. 10 shows a first position 1010 at a first point in time when the user's right foot contacts the ground, a second position 1020 at a second point in time when the user's left foot contacts the ground after the first point in time, a third position 1012 at a third point in time when the right foot contacts the ground again after the second point in time, and a fourth position 1022 at a fourth point in time when the left foot contacts the ground again after the third point in time. The length from the first location 1010 where the user's right foot contacts the ground to the second location 1020 where the user's left foot contacts the ground may correspond to the left step size of the user, and the length from the second location 1020 where the user's left foot contacts the ground to the third location 1012 where the user's right foot contacts the ground may correspond to the right step size of the user. Both the left and right steps may correspond to a one-step size, and the stride length from when the right foot (or left foot) contacts the ground to when the right foot (or left foot) contacts the ground again may correspond to a two-step stride length. The stride length of the two steps may correspond to a sum of the left and right stride lengths.
Fig. 11a and 11b each show a diagram of an operation of estimating a stride length based on hip angle values according to an embodiment.
Referring to fig. 11a and 11b, when calculating the ratio of the stride length of two steps occupied by each of the right and left steps, the right and left steps may be estimated.
The right step size may be determined based on fig. 11a and equation 1 below.
Equation 1
Right step = l× (sin θ L+sinθR)
In this case, L may correspond to the right leg length and the left leg length from the position 1110 of the user's hip joint. θ L may be the hip angle when the hip angle of the user's left leg reaches a rearward maximum of the hip angle values during walking of the user, where the hip angle value is formed by the user's left leg and a straight line 1105 perpendicular to the ground and passing through the location 1110 of the hip. θ R may be the hip angle when the hip angle of the user's right leg reaches a forward maximum of the hip angle values during walking of the user, where the hip angle values are formed by the user's right leg and the straight line 1105. Both θ L and θ R are assumed to have positive values.
The left step size may be determined based on fig. 11b and equation 2 below.
Equation 2
Left step = l× (sin θ L′+sinθR')
In this case, L may correspond to the right leg length and the left leg length from the position 1110 of the user's hip joint. θ ' L may be the hip angle when the hip angle of the user's left leg reaches a forward maximum of the hip angle values during walking of the user, where the hip angle values are formed by the user's left leg and a straight line 1105 perpendicular to the ground and passing through the location 1110 of the hip. θ ' R may be the hip angle when the hip angle of the user's right leg reaches the rearward maximum of the hip angle values formed by the user's right leg and the straight line 1105 during walking of the user. Both θ 'L and θ' R are assumed to have positive values.
The right step size may be determined based on the two-step stride length and the hip angle value, as shown in equation 3 below.
Equation 3
The left step size may be determined based on the two-step stride length and the hip angle value, as shown in equation 4 below.
Equation 4
A wearable device (e.g., the wearable device 100 of fig. 1 and 3 or the wearable device 100 of fig. 2) may estimate a right step size and a left step size from a two-step stride length and hip angle value by using equations 3 and 4.
Fig. 12 is a diagram showing a screen for providing gait evaluation information according to an embodiment.
Referring to fig. 12, the electronic device 210 may provide a UI screen 1210 for providing gait evaluation information of the user. The gait assessment information may include information regarding one or more estimated gait indices and a gait score determined based on the gait indices. In an embodiment, the electronic device 210 may provide a gait index including at least one of a total number of steps, a spent walking time, an (average) walking speed, a total walking distance, an (average) step/stride length, a gait symmetry index, a gait change index, and a walking ratio through the UI screen 1210.
In embodiments, the electronic device 210 may update the average step size/stride length of one or more steps in real time and provide the updated average step size/stride length to the user through the UI screen 1210, or may provide voice data for informing the average step size/stride length to the user through the audio device. In an embodiment, the electronic device 210 may calculate a relative percentile score for each of the gait indices including the walking speed, walking ratio, gait symmetry index and gait variation index in the user group to which the user's age or gender belongs, and may provide the user with a weighted average or calculated percentile score of the average for each gait index. The electronic device 210 may collect gait indices of a user group of an age group similar to the user's age and may calculate a percentile position of the user's gait indices in the user group. For example, it is assumed that when the walking speed of a 65-year-old user is fastest compared to the walking speed of a user group of 60 to 70 age groups to which the user's age belongs, the percentile score of the user is 100, and when the walking speed of the user is slowest, the percentile score is 0, and the percentile score corresponding to the user can be determined based on which percentile position the walking speed of the user corresponds to.
Fig. 13 is a diagram showing the configuration and operation of a training apparatus for training a walking speed estimation model according to an embodiment.
Referring to fig. 13, the training apparatus 1300 may be an apparatus for training a walking speed estimation model by a linear regression method or a machine learning method. The training device 1300 may train the walking speed estimation model based on sensor data and actual walking speed data of the wearable device 100 obtained during the user's walking.
Exercise device 1300 may include a processor 1310 and a memory 1320. Processor 1310 may control at least one other component (e.g., a hardware or software component) of exercise device 1300 and may perform various types of data processing or operations. The processor 1310 may include a main processor (e.g., a CPU or AP) or may be a secondary processor (e.g., GPU, NPU, ISP, a sensor hub processor, or a CP) that operates independently of or in conjunction with the main processor. Memory 1320 may store various data used by at least one component of exercise device 1300 (e.g., processor 1310). Memory 1320 may include at least one instruction executable by processor 1310. Memory 1320 may include volatile memory or nonvolatile memory.
The processor 1310 may process (e.g., pre-process) the collected training data to train the walking speed estimation model. For example, the training data may include speed data of the treadmill when the user is caused to walk on the treadmill at various walking speeds (e.g., 0km/h, 0.5km/h, 1.0km/h, etc.) while wearing the wearable device 100 and sensor data obtained by sensors (e.g., angle sensors or IMUs) of the wearable device 100 when the user is walking at a particular walking speed. The processor 1310 may remove noise from the collected training data that removes sensor data obtained during times other than when the user is walking at a particular walking speed and sensor data obtained while the user is stationary. The processor 1310 may perform normalization for each type of sensor data on the noise-removed training data.
The processor 1310 may select the type of walking speed estimation model for performing the training. For example, a CNN model, a linear regression model, or a model in which a full connection layer is connected to an output unit of an LSTM model may be used as the walking speed estimation model, but examples are not limited to the foregoing models.
The processor 1310 may select input features to be used for walking speed estimation model training. The processor 1310 may specify input data for a walking speed estimation model. For example, the processor 1310 may select as input features angular velocity data (e.g., each of the x-axis, y-axis, and z-axis) and acceleration data (e.g., each of the x-axis, y-axis, and z-axis acceleration data) obtained from the IMU of the wearable device 100, or may select as input features angular velocity data (e.g., angular velocity data corresponding to the right hip joint or angular velocity data corresponding to the left hip joint) of the hip joint obtained from an angle sensor of the wearable device 100. Alternatively, the processor 1310 may select acceleration data, angular velocity data, rotation angle data of the hip joint (e.g., joint angle of the right hip joint or joint angle of the left hip joint) and angular velocity data obtained from the IMU as input features.
The processor 1310 may input the selected input features to the walking speed estimation model, may calculate a difference between an actual walking speed value (e.g., the speed of the treadmill) and a walking speed estimation value output from the walking speed estimation model, and may update parameters of the walking speed estimation model so that the difference may be reduced. For example, the processor 1310 may perform the update process by an error back propagation algorithm. The processor 1310 may repeatedly perform the training process of the walking speed estimation model up to the turn determined for training. Parameters of the walking speed estimation model may be updated at each round.
When the training process is completed, the updated parameter walking speed estimation model may be embedded in the electronic device 210 and/or the wearable device 100. The walking speed estimation model may estimate the user's walking speed in real time based on sensor data corresponding to the type of input features selected during training of the walking speed estimation model from among various types of sensor data measured in the IMU and angle sensors of the wearable device 100.
Fig. 14 is a flowchart illustrating an operation of a training method of the walking speed estimation model according to the embodiment. According to an embodiment, the training method may be performed by a training apparatus (e.g., training apparatus 1300 of fig. 13).
Referring to fig. 14, in operation 1410, training data for walking speed estimation model training may be collected. In an embodiment, sensor data obtained from the angle sensors and IMU of the wearable device 100 may be collected as a user wearing the wearable device 100 walks at various walking speeds on the treadmill. For training data collection, the user may walk on the treadmill at each speed: 0km/h, 0.5km/h, 1.0km/h, …, 6.5km/h, 7.0km/h by 10 steps. Training data may be collected in each operational state of the wearable device 100 (e.g., a state in which a walking assist mode is active, a state in which an exercise assist mode is active, or a state in which a walking assist mode or an exercise assist mode is inactive).
Sensor data may be collected at each sampling time (e.g., 10 ms). For example, the collected sensor data may include speed data of the treadmill corresponding to an actual walking speed of the user, torque data regarding a torque value generated by a driving module of the wearable device 100, sensor data of an angle sensor of the wearable device 100 (e.g., angular speed data corresponding to a right hip joint, angular speed data corresponding to a left hip joint, a joint angle of a right hip joint, or a joint angle of a left hip joint), or sensor data of an IMU of the wearable device 100 (e.g., acceleration data of each of an x-axis, a y-axis, and a y-axis, gyroscope sensor data, or euler angle data).
In operation 1420, the training device may pre-process the collected training data. The training device may perform noise removal (or outlier cancellation) that removes, from the collected training data, sensor data obtained during times other than when the user is walking at a particular walking speed and sensor data obtained while the user is stationary. The training device may perform data normalization on the noise-removed training data. For example, the training device may calculate an average value and a standard deviation of each type of data included in the training data, and may normalize the data of each data type by using the calculated average value and standard deviation.
In operation 1430, the training device may train the walking speed estimation model based on the training data on which the preprocessing is performed. For example, a CNN model, a linear regression model, or a model in which a full connection layer is connected to an output unit of an LSTM model may be used as the walking speed estimation model, but examples are not limited to the foregoing models. The model of the fully connected layer connected to the output unit of the LSTM model may be a model implemented in such a way: the walking speed estimation value in the hidden state inside the LSTM model may be output to the outside through the full connection layer connected to the LSTM model.
The training device may select, as an input feature, sensor data to be used for training of the walking speed estimation model, among various types of sensor data obtained from the wearable device 100. For example, any one of (1) sensor data of the IMU, (2) sensor data of the angle sensor, or (3) sensor data of the IMU and sensor data of the angle sensor may be selected as the input feature.
The training device may initially determine the hyper-parameters for training of the walking speed estimation model. For example, the determined hyper-parameters may include a loss type (e.g., mean square error), a learning rate, a turn, a number of hidden states of the walking speed estimation model, a batch size, and so forth.
The training device may repeatedly perform up to a round determined based on the training data. During training, the training device may update parameters (e.g., weights) of the walking speed estimation model at each round. For example, the training device may calculate a difference by comparing an actual walking speed value with the walking speed estimation value output by the walking speed estimation model according to the backward error propagation algorithm, and may update the parameters of the walking speed estimation model so that the difference may be reduced.
In an embodiment, the training device may perform a verification process of the training result when training of the walking speed estimation model is completed. For example, the training device may divide the entire data into 10 groups, may use the data of one of the groups as verification data, and may use the data of the remaining 9 groups as training data. The training device may perform training based on the training data, and then, may obtain a walking speed estimation value by inputting the verification data to the walking speed estimation model, and may compare the obtained walking speed estimation value with an actual walking speed value. The training device may determine the accuracy of the walking speed estimation model based on the comparison result. As the walking speed estimate is more similar to the actual walking speed value, accuracy may be determined to be higher. When the accuracy of the walking speed estimation model is lower than the required level, the training device may perform the training and verification process again by changing the selection of input features input to the walking speed estimation model or changing the super-parameters. As a result of the verification, when the accuracy of the walking speed estimation model satisfies the required level, the training device may perform the training by using the super parameters for the training, with the entire data (e.g., 10 sets of data) as the training data.
In operation 1440, the walking speed of the user may be estimated by using the trained walking speed estimation model. When the training process is completed, the walking speed estimation model may estimate the user's walking speed based on sensor data corresponding to the type of input features selected during the training process of the walking speed estimation model from among various types of sensor data measured from the IMU and the angle sensor of the wearable device 100. When the walking speed estimation model is a model in which a full connection layer is connected to an output unit of the LSTM model, a vector of the selected input feature of the wearable apparatus 100, which is obtained at time t, may be input to the walking speed estimation model, and when the vector of the input feature is applied to a weight matrix of the LSTM model, which is determined through a training process, the walking speed estimation value may be finally output from the walking speed estimation model at time t. The estimation of the walking speed may be performed in the electronic device 210 and/or the wearable device 100.
As described above, the process of estimating gait index described herein may also be performed in the wearable device 100 and the electronic device 210. For example, the processor 512 of the wearable device 100 may estimate the walking speed of the user in real time by inputting the sensor data obtained by the sensor modules 520 and 520-1 to the walking speed estimation model. In this case, the sensor data may include at least one of a hip joint angle value (e.g., angular velocity or rotational velocity) measured by the angle sensor 125 and a motion value (e.g., acceleration or rotational velocity) measured by the IMU135 of the user. The wearable device 100 may send the estimated walking speed to the electronic device 210 through the communication module 516.
It should be understood that the various embodiments of the disclosure and the terminology used therein are not intended to limit the technical features set forth herein to the particular embodiments, and include various modifications, equivalents, or alternatives to the corresponding embodiments. The same reference numerals may be used for similar or related parts in connection with the description of the drawings. It is to be understood that the singular form of a noun corresponding to an item may include one or more things unless the context clearly indicates otherwise. As used herein, each of "a or B", "at least one of a and B", "at least one of a or B", "A, B or C", "at least one of A, B and C", and "A, B or C" may include any one of the items listed together in the respective phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from other components in the discussion and to not otherwise (e.g., importance or order) limit the component. It will be understood that if an element (e.g., a first element) is referred to as being "coupled to," "connected to," or "connected to" another element (e.g., a second element) with or without the term "operatively" or "communicatively," then the element can be directly (e.g., through a wire), wirelessly, or via a third element.
As used in connection with various embodiments of the present disclosure, the term "module" may include an element implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component, or a minimal unit or portion thereof, adapted to perform one or more functions. For example, according to an embodiment, a module may be implemented in the form of an Application Specific Integrated Circuit (ASIC).
The software may include a computer program, a piece of code, instructions, or some combination thereof, to individually or collectively instruct or configure the processing device to operate as desired. The software and data may be permanently or temporarily embodied in any type of machine, component, physical or virtual device or computer storage medium or apparatus capable of providing instructions or data to or for interpretation by a processing apparatus. The software may also be distributed over network-coupled computer systems so that the software is stored and executed in a distributed fashion. The software and data may be stored by one or more non-transitory computer readable recording media. The embodiments set forth herein may be implemented as software, where the software includes one or more instructions stored in a machine-readable storage medium (e.g., memory 514). For example, a processor of the machine may invoke at least one of the one or more instructions stored in the storage medium and execute it. This allows the machine to be operated on in accordance with the at least one instruction invoked to perform at least one function. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" means only that the storage medium is a tangible device and does not include a signal (e.g., electromagnetic waves), but the term does not distinguish whether the data is semi-permanently stored in the storage medium or temporarily stored in the storage medium.
According to one embodiment, a method according to embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be used as a product to conduct transactions between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium, such as a compact disc read only memory (CD-ROM), or distributed online (e.g., downloaded or uploaded) through an application store, such as PlayStore TM, or distributed directly between two user devices, such as smartphones. If distributed online, at least a portion of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium, such as a memory of a manufacturer's server, an application store's server, or a relay server.
According to embodiments, each of the above-described components (e.g., a module or a program) may include a single entity or a plurality of entities, and some of the plurality of entities may be separately provided in different components. Depending on the embodiment, one or more of the above components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to an embodiment, the integrated components may still perform one or more of the functions of each of the components in the same or similar manner as the one or more functions were performed by a corresponding one of the pre-integrated components. According to embodiments, operations performed by a module, a program, or another component may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be performed in a different order or omitted, or one or more other operations may be added.