Skip to content

Unify RoPE strategies #388

Description

@jonatanklosko

There has been some effort in hf/transformers to extract and share RoPE types across models, so we should be able to unify more, especially the loading/conversion parts. Some of the improvements are still in progress. A few relevant PRs:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:choreInternal improvements

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions