Repository navigation
Commit 3571bfa
authored
[HDX-3999] Implement hdx query with connections command and Drain pattern mining (#2133)
## Summary
Implements [HDX-3999](https://linear.app/clickhouse/issue/HDX-3999/implement-hdx-query-one-shot-clickhouse-sql-execution) — a one-shot `hdx query` for executing raw ClickHouse SQL against a configured connection, plus supporting UX around it for agent-driven workflows.
This is pre-launch iteration on the unreleased CLI surface; no released behavior changes.
## What changed
### `hdx connections` (new subcommand)
- Lists ClickHouse connections configured for the authenticated team (`id`, `name`, `host`), matching the UX of `hdx sources` and `hdx dashboards`.
- Supports `--json` for programmatic consumption (`hdx connections --json | jq -r '.[0].id'`).
- Exports `ConnectionResponse` from `api/client.ts` for consistency with other response types.
- Registered in `packages/cli/AGENTS.md`.
### `hdx query` reshape
- Replaces `--source <nameOrId>` with `--connection-id <id>`. Raw SQL only needs a connection, so the source-to-connection lookup was unnecessary indirection. Agents discover IDs via `hdx connections --json`.
- On query error, lazily fetches the connection list to distinguish "unknown connection ID" from "bad SQL" — no happy-path cost.
- Default `--format` changes from `JSON` (wrapped envelope) → `JSONEachRow` (NDJSON). Streamable, smaller, trivially consumable from shell pipelines. Envelope is still available via `--format JSON`.
- Documents the exit-code contract in `--help` so consumers can disambiguate empty result (exit 0, empty stdout) from failure (exit 1, stderr message).
### `hdx query --patterns` (Drain-based pattern mining)
- `--patterns`: clusters the result's text into log patterns via Drain (`@hyperdx/common-utils/dist/drain`) and emits one NDJSON object per pattern sorted by count desc:
```json
{"pattern":"<template>","count":<n>,"sample":"<first sample>"}
```
- `--body-column <name>`: cluster only that column's string value. Defaults to clustering the whole row JSON-serialized, which works for any SELECT shape. If the column isn't in the result, exits 1 with the available columns on stderr.
- Pattern mining forces `JSON` internally regardless of `--format` so named rows can be iterated; a one-line note is written to stderr if the user passed a conflicting `--format`.
- The existing raw-SQL path is unchanged when `--patterns` is not set.
### Help-text polish
- Top-level `hdx --help` now mentions `--patterns` in the `query` description so agents can discover it without drilling into `hdx query --help`.
- Expanded help block documents `--body-column` in detail and adds a sampling tip with an explicit performance warning that `ORDER BY rand()` scans and sorts all WHERE-matched rows before LIMIT and must be paired with a selective WHERE.
## Why
Agents driving HyperDX need a direct, bounded way to run SQL and summarize large result sets. `--patterns` makes it cheap to ask "what are the top log templates in this window" without a separate pipeline step.
## Testing
- Manual: `hdx connections`, `hdx connections --json`, `hdx query --connection-id <id> --sql "SELECT 1"`, `hdx query --connection-id <id> --patterns --body-column Body --sql "SELECT Body FROM default.otel_logs LIMIT 1000"`.
- `make ci-lint` passes.
- No automated tests added — `packages/cli` has no Jest suite for `cli.tsx` today; following the existing convention.
## Breaking changes
- `hdx query` no longer accepts `--source`. The command is pre-launch so no downstream consumers yet.1 parent 2a28aee commit 3571bfa
3 files changed
Lines changed: 245 additions & 33 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
14 | 18 | | |
15 | 19 | | |
16 | 20 | | |
| |||
664 | 668 | | |
665 | 669 | | |
666 | 670 | | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
667 | 731 | | |
668 | 732 | | |
669 | 733 | | |
| |||
784 | 848 | | |
785 | 849 | | |
786 | 850 | | |
787 | | - | |
788 | | - | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
789 | 858 | | |
790 | 859 | | |
791 | | - | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
| 868 | + | |
792 | 869 | | |
793 | 870 | | |
794 | 871 | | |
795 | 872 | | |
796 | 873 | | |
797 | | - | |
798 | | - | |
| 874 | + | |
| 875 | + | |
799 | 876 | | |
800 | | - | |
801 | | - | |
| 877 | + | |
| 878 | + | |
| 879 | + | |
802 | 880 | | |
803 | 881 | | |
804 | | - | |
805 | | - | |
806 | | - | |
807 | | - | |
808 | | - | |
| 882 | + | |
| 883 | + | |
| 884 | + | |
| 885 | + | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
809 | 928 | | |
810 | 929 | | |
811 | | - | |
812 | | - | |
813 | | - | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
814 | 935 | | |
815 | 936 | | |
816 | 937 | | |
817 | 938 | | |
| 939 | + | |
818 | 940 | | |
819 | | - | |
820 | | - | |
821 | | - | |
822 | | - | |
823 | | - | |
824 | | - | |
825 | | - | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
826 | 965 | | |
827 | | - | |
828 | | - | |
829 | | - | |
830 | | - | |
831 | | - | |
832 | | - | |
833 | | - | |
834 | | - | |
| 966 | + | |
| 967 | + | |
| 968 | + | |
| 969 | + | |
| 970 | + | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
| 978 | + | |
835 | 979 | | |
836 | | - | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
| 984 | + | |
| 985 | + | |
| 986 | + | |
| 987 | + | |
| 988 | + | |
| 989 | + | |
| 990 | + | |
| 991 | + | |
| 992 | + | |
| 993 | + | |
| 994 | + | |
| 995 | + | |
| 996 | + | |
| 997 | + | |
| 998 | + | |
| 999 | + | |
| 1000 | + | |
837 | 1001 | | |
838 | | - | |
| 1002 | + | |
| 1003 | + | |
| 1004 | + | |
| 1005 | + | |
| 1006 | + | |
| 1007 | + | |
| 1008 | + | |
| 1009 | + | |
| 1010 | + | |
| 1011 | + | |
| 1012 | + | |
| 1013 | + | |
| 1014 | + | |
| 1015 | + | |
| 1016 | + | |
| 1017 | + | |
839 | 1018 | | |
840 | 1019 | | |
841 | 1020 | | |
842 | | - | |
| 1021 | + | |
843 | 1022 | | |
844 | 1023 | | |
845 | 1024 | | |
| |||
849 | 1028 | | |
850 | 1029 | | |
851 | 1030 | | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
| 1042 | + | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
| 1048 | + | |
| 1049 | + | |
| 1050 | + | |
| 1051 | + | |
| 1052 | + | |
852 | 1053 | | |
853 | 1054 | | |
854 | 1055 | | |
| |||
0 commit comments