Skip to content

apply_fsdp is not applied to the VLM projector #31

@tomiock

Description

@tomiock

With FSPD2, it should be applied to all the modules present in the highest level module. Currently it's only applied to the encoder, layer norms, token embeddings and decoder, the projector (simple MLP) should have the function called on it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghpc_preptask needed to be completed before HPCtraining

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions