Skip to content

Proposal: Filters and big data API #17

@davisjam

Description

@davisjam

This is a proposal, not a bug report. As such it's a bit unfocused, mostly brainstorming.

  • Filter the package.jsons from meta/
  • Filter the .tgzs from current/, optionally based on the filtered package.json's from meta/

This is similar to the code search and AST execute APIs, but seems more general-purpose.

I know I can grep through the files in pool/ for this, but that's brittle because it depends on Gzemnid internals. Can Gzemnid expose this facility?

Also, Gzemnid queries are a big data problem. Thoughts on exposing some kind of producer-consumer web API so I can easily distribute queries across a cluster of workers?

I'm happy to work on a PR if we can decide on the API (e.g. celery or similar).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions