Added an HeuristicPolicy with Control Lyapunov Controller#74
Added an HeuristicPolicy with Control Lyapunov Controller#74matteobettini merged 12 commits intoproroklab:mainfrom
Conversation
…igation scenario.
matteobettini
left a comment
There was a problem hiding this comment.
Thanks so much for this!
I left some initial comments.
My main question is. Is there a roadmap you see that we can take to make this type of controllers generally available to any vmas task and not specific to navigation?
Co-authored-by: Matteo Bettini <55539777+matteobettini@users.noreply.github.com>
Co-authored-by: Matteo Bettini <55539777+matteobettini@users.noreply.github.com>
Co-authored-by: Matteo Bettini <55539777+matteobettini@users.noreply.github.com>
Co-authored-by: Matteo Bettini <55539777+matteobettini@users.noreply.github.com>
Co-authored-by: Matteo Bettini <55539777+matteobettini@users.noreply.github.com>
|
Yes, this type of controller has two purposes once Control Barrier Functions are included: (1) It can steer agents to desired goals. (2) It can be used as a safety shielding (For example, to avoid collisions or never exceed certain relative velocities with respect to other agents). With (2), the controller can take an action from another policy and use it as a reference. The safety shielding will only be activated if the safety constraints are violated. This feature can be used for safe RL learning. |
vmas/scenarios/navigation.py
Outdated
|
|
||
|
|
||
| class HeuristicPolicy(BaseHeuristicPolicy): | ||
| def __init__(self, ctrl_params=[0.2, 1.0, 1.0, 100.0], *args, **kwargs): |
There was a problem hiding this comment.
Here could we have clf_epsilon=0.2, cbf_eta=1.0 etc. ?
There was a problem hiding this comment.
The clf_epsilon defines the convergence rate of the Lyapunov function. Let me remove cbf parameters for now because it's not implemented yet.
There was a problem hiding this comment.
I just meant having each parameter separate instead of a list.
So that users can change just one.
There was a problem hiding this comment.
I see. One moment, let me modify it.
Ok so here is what we can do. As I understand it, this heuristic provides an example of (1). We can merge this as a heuristic for navigation as it is specific to goal following. Regarding (2), it would be nice to have a customizable component in https://github.com/proroklab/VectorizedMultiAgentSimulator/tree/main/vmas/simulator/controllers that takes some custom constraints and applies the CBF to the agent action. This compenents will then be called in A challenge here is how to create such a component so that it is able to take general constraints. |
matteobettini
left a comment
There was a problem hiding this comment.
Amazing, could you format with black 23.1.0
Co-authored-by: Matteo Bettini <55539777+matteobettini@users.noreply.github.com>
QP controller with CLF.