You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The actions agent.action.u (which before corresponed to agent.state.force) are now detached from the physics. Dynamics will read actions and write into agent.state.force and agent.state.torque. This allows users to play around with action sizes and new dynamics more flexibly.
At agent construction you can specify the agent dynamics (Agent(dynamics=Holonomic()) is the default). There are many available (Holonomic,HolonomicWithRotation,DiffDrive,KinematicBicycle)
The dynamics will define the action size. You can also override this and define an action size yourself Agent(action_size=3). But you have to provide at least the minimum number of actions required by the dynamics (dynamics.needed_action_size). The dynamics will use the first dynamics.needed_action_size actions and the rest can be used for other user needs
The discrete actions will have 3 options for each agent.action_size action. By default the holonomic agent has 2 actions: force_x (with 3 options stay,decrement,increment) and force_y (with 3 options stay,decrement,increment). Holonomic agent with rotations have 3 actions, with the additional one controlling torque (with the 3 options stay, decrement, increment). And so on for other dynamics....
Action ranges, multipliers and noise can be controlled at agent construction with u_range, u_multiplier and u_noise. These can either be a scalar or a list. In case of a list, the list must have length agent.action_size
BC-BREAKING Changes
Removed u_rot_range,u_rot_multiplier in favor of a more general u_range, u_multiplier and u_noise which can be a list (one elem per action )
Removed MultiDiscrete actions. While before in case of multiple actions there was a space Multidiscrete([action_1_size, action_2_size]) , now there is a space DIscrete(action_1_size * action_2_size). It is still possible to ask MultiDIscrete action spaces by setting vmas.make_env(multidiscrete_actions=True)
Discrete actions for the default dynamics (holonomic no rotation) have changed from 5 (up,down,left,right,stay) to 9, that now include the corners.
matteobettini
changed the title
[Feature] Actions detched from physics and any number of actions
[Feature] Actions detached from physics and any number of actions
Jan 8, 2024
matteobettini
changed the title
[Feature] Actions detached from physics and any number of actions
[Feature] Actions detached from physics and allow any number of actions
Jan 8, 2024
A small feedback: in use_vmas_env.py, the default value of variable simple_2d_action is [0, -1.0] if continuous_actions = True. This would cause an error if you set scenario_name to kinematic_bicycle, in which the u_range is set such that the maximum allowed steering angle is 30 degree (around 0.524 in radius). To avoid this error, the absolute value of the second entry of simple_2d_action should be smaller than 0.524. For example, setting [0, -0.5] as the default value for simple_2d_action could avoid the error.
A small feedback: in use_vmas_env.py, the default value of variable simple_2d_action is [0, -1.0] if continuous_actions = True. This would cause an error if you set scenario_name to kinematic_bicycle, in which the u_range is set such that the maximum allowed steering angle is 30 degree (around 0.524 in radius). To avoid this error, the absolute value of the second entry of simple_2d_action should be smaller than 0.524. For example, setting [0, -0.5] as the default value for simple_2d_action could avoid the error.
Got it. Will make sure the simple actions are feasible in all scenarios
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New features
agent.action.u(which before corresponed toagent.state.force) are now detached from the physics. Dynamics will read actions and write intoagent.state.forceandagent.state.torque. This allows users to play around with action sizes and new dynamics more flexibly.Agent(dynamics=Holonomic())is the default). There are many available (Holonomic,HolonomicWithRotation,DiffDrive,KinematicBicycle)Agent(action_size=3). But you have to provide at least the minimum number of actions required by the dynamics(dynamics.needed_action_size). The dynamics will use the firstdynamics.needed_action_sizeactions and the rest can be used for other user needsagent.action_sizeaction. By default the holonomic agent has 2 actions: force_x (with 3 options stay,decrement,increment) and force_y (with 3 options stay,decrement,increment). Holonomic agent with rotations have 3 actions, with the additional one controlling torque (with the 3 options stay, decrement, increment). And so on for other dynamics....u_range,u_multiplierandu_noise. These can either be a scalar or a list. In case of a list, the list must have lengthagent.action_sizeBC-BREAKING Changes
u_rot_range,u_rot_multiplierin favor of a more generalu_range,u_multiplierandu_noisewhich can be a list (one elem per action )Multidiscrete([action_1_size, action_2_size]), now there is a spaceDIscrete(action_1_size * action_2_size). It is still possible to ask MultiDIscrete action spaces by settingvmas.make_env(multidiscrete_actions=True)FIxes #51
Fixes #75
cc @Jianye-Xu @Zartris