Skip to content

Skill/tool usage drops off as a project matures — lc/astra skills stop being auto-invoked later in a project #149

Open

Description

@calumhrmurray

What happened

In the early stages of a project the lc/astra skills and tools are invoked
consistently and effectively. As the project matures, Claude increasingly stops
reaching for them on its own — work that should route through a skill gets done
ad-hoc instead.

Concrete example from a live session: the user asked Claude to review a paper,
and Claude did the work without invoking the relevant ASTRA skill
(paper-extraction / the astra paper workflow). It hand-rolled the steps
(WebFetch the abstract, manual quote extraction, astra paper add --pdf) rather
than going through the skill path. The end result was fine, but the skill that
exists for exactly this was bypassed.

Impact

The skills encode the correct, provenance-tracked workflow. Skipping them later in
a project means inconsistent process (manual vs skill-driven), and the value of
having the skills degrades precisely when the project is most complex and most
needs the guardrails.

Possible direction (not a prescription)

  • Skill-selection seems to weaken as session/project context grows — worth checking
    whether long-context or accumulated CLAUDE.md state is crowding out the
    skill-trigger reminders.
  • A periodic "did a skill match this request?" check, or stronger trigger phrasing
    for paper/spec/recipe operations later in a project, might help.

Environment

  • ASTRA: 0.2.9
  • lightcone-cli: 0.3.6
  • Python: 3.12.5
  • OS: Linux 4.18.0-553 (EL8)

This was written by Claude.

Activity

  1. calumhrmurray commented on Jun 8, 2026

    @calumhrmurray
    Author

    Folding in #151 as a concrete instance of this same root cause (skills not auto-invoked in a mature project).

    Instance: in an existing ASTRA project, the user asked Claude to "read some papers to check we are not missing anything." Claude ran ad-hoc web search instead of routing through the ASTRA literature flow, and wrote nothing back into the spec.

    Specific expected routing this adds (not in the original report): a literature-review request in an existing project should mirror lc-new behavior —

    • invoke the astra skill for the prior_insights/evidence schema,
    • use the lc-extractor agent to pull verified quotes from the papers,
    • land them as structured prior_insights in astra.yaml, each attached to the decision it bears on.

    Note the project's astra.yaml had no prior_insights section at all, so there was no existing literature layer to extend — the ad-hoc path produced unverified paraphrases (one WebFetch even failed to extract the real content) with no provenance.

    Closing #151 as a duplicate of this umbrella issue.

    This comment was written by Claude on behalf of the user.

  2. aboucaud commented on Oct 6, 2026

    @aboucaud
    Member

    @cailmdaley is this relevant to move this issue to the agent-skills repo or shall we close it ?

  3. cailmdaley commented on Oct 12, 2026

    @cailmdaley
    Member

    yes i think it's still relevant @aboucaud and i agree it should be moved. i still notice the lightcone skill rarely gets activated unless i explicitly ask. we could have an eval that emulates a mature project and makes sure the skill gets activated automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions