Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 11 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,6 @@ request := gush.FlowRequest{Flow: Notify, Input: receipt}
err := ctx.RunFlow(request)
```

`WithCron("0 2 * * *")` runs a flow on a schedule in UTC until it is canceled, terminated or stopped by `WithTimeout`. Start it with `StartFlow`, because `RunFlow` returns an error: a schedule has no result to wait for. A client's `StartFlow` returns an error for an invalid expression. On a child flow, Temporal instead fails the parent's workflow task, which retries until the parent's `WithTimeout`, so check an expression that comes from outside first.

Flows and steps are registered under their bare Go function names, and can be called by that name as a string. Two functions with the same name in different packages collide, and `Serve` reports it. Use `NewFlowHandler` and `NewStepHandler` to choose a name, which closures and the instantiations of a generic function need too.

`string` and `[]byte` travel as raw bytes and everything else as JSON, so read a value back as the type it was written.
Expand Down Expand Up @@ -156,10 +154,9 @@ A step that is already running only finds out about the cancellation through `ct
| `WithFairKey(key)` | both | Group to schedule fairly against, such as a tenant. |
| `WithLogAttribute(k, v)` | both | Attribute on every log line of this run and its children. |
| `WithLogAttributes(m)` | both | The same, for several attributes. |
| `WithID(id)` | flow | Flow ID. Otherwise one is generated, which `StartFlow` returns. A client's `RunFlow` or `StartFlow` joins a flow already running with this ID and ignores this run's input, and a child flow gets an error. `RunFlow` then waits for that flow, even across a `WithCron` schedule. |
| `WithTimeout(d)` | flow | Limit on the whole execution, including every retry and every scheduled run. |
| `WithCron(expr)` | flow | Schedule in UTC. Use with `StartFlow`. |
| `WithConfig(c)` | flow | Starts from a `FlowConfig`, such as one decoded from a queue. Fields left out of the JSON keep their defaults. Whoever applies it chooses `RunFlow` or `StartFlow`, and one with a cron schedule needs `StartFlow`. |
| `WithID(id)` | flow | Flow ID. Otherwise one is generated, which `StartFlow` returns. A client's `RunFlow` or `StartFlow` joins a flow already running with this ID and ignores this run's input, and a child flow gets an error. |
| `WithTimeout(d)` | flow | Limit on the whole execution, including every retry. |
| `WithConfig(c)` | flow | Starts from a `FlowConfig`, such as one decoded from a queue. Fields left out of the JSON keep their defaults. Whoever applies it chooses `RunFlow` or `StartFlow`. |
| `WithQueue(q)` | step | Runs on another queue. |
| `WithAttemptTimeout(d)` | step | Limit on each attempt, not the step as a whole. |
| `WithHeartbeatTimeout(d)` | step | Fails an attempt that stops calling `ctx.Heartbeat()`. |
Expand Down Expand Up @@ -282,13 +279,14 @@ Lines written while Temporal replays a flow are dropped, so each appears once. H

### Export over OTLP

`gush.Dial` starts exporting when an endpoint is set, and `Close` flushes. A process that logs without a client calls `gush.InitLogExport()` itself. The first variable set here wins:
`gush.Dial` starts exporting when `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` or `OTEL_EXPORTER_OTLP_ENDPOINT` is set, and `Close` flushes. A process that logs without a client calls `gush.InitLogExport()` itself. [OpenTelemetry's other variables](https://opentelemetry.io/docs/specs/otel/protocol/exporter/) set the headers and TLS, and the protocol is `http/protobuf` unless they ask for `grpc`:

```sh
OTEL_EXPORTER_OTLP_ENDPOINT=https://collector:4317
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
```

| Variable | Protocol |
|---|---|
| `OTEL_ENDPOINT` | OTLP/gRPC. A bare `host:port` is plaintext; a URL's scheme chooses, and `https://` means TLS. |
| `OTEL_HTTP_ENDPOINT` | OTLP/HTTP, as an `http://` or `https://` URL. Logs go to `/v1/logs`. |
| `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`, `OTEL_EXPORTER_OTLP_ENDPOINT` | The [standard OpenTelemetry variables](https://opentelemetry.io/docs/specs/otel/protocol/exporter/), with their protocol, header and TLS settings. |
The endpoint must be a URL, or `Dial` returns an error. `https://` is TLS and `http://` is plaintext, and `OTEL_EXPORTER_OTLP_INSECURE` does not change that. Give the port too, because leaving it out does not select OTLP's 4317 or 4318. Over HTTP, `/v1/logs` is added to `OTEL_EXPORTER_OTLP_ENDPOINT`, but `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` is used as written, so it needs the full path, such as `http://collector:4318/v1/logs`.

An application with its own logging pipeline keeps control: a provider set with OpenTelemetry's `global.SetLoggerProvider` always wins, and Gush uses it instead of starting an export. Importing Gush never changes that global. With no provider and no endpoint, logs go to standard error, as do the Temporal SDK's own messages unless `Options.Logger` says otherwise.

Expand Down Expand Up @@ -328,7 +326,7 @@ options := gush.Options{
c, err := gush.Dial(options)
```

An empty `HostPort` falls back to `TEMPORAL_ENDPOINT`, then `TEMPORAL_ADDRESS`, then `localhost:7233`. An empty `Namespace` falls back to `TEMPORAL_NAMESPACE`, then the default namespace.
An empty `HostPort` falls back to `TEMPORAL_ADDRESS`, then `localhost:7233`. An empty `Namespace` falls back to `TEMPORAL_NAMESPACE`, then the default namespace.

The package-level functions, such as `gush.RunFlow` and `gush.Serve`, use a shared client configured from those variables. They suit a process that talks to one server.

Expand Down
5 changes: 3 additions & 2 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,6 @@ vulnerabilities even if you cannot reproduce them on the latest release.
stores them in its history. Do not put secrets in them.
- Resume tokens let their holder finish a suspended step. Treat them as credentials
when handing them to another system.
- `OTEL_ENDPOINT` set to a bare `host:port` sends logs in plaintext. Use an
`https://` URL when logs leave a trusted network.
- With no certificate variable set, an `http://` value in `OTEL_EXPORTER_OTLP_ENDPOINT`
or `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` sends logs in plaintext. Use an `https://` URL
when logs leave a trusted network.
8 changes: 4 additions & 4 deletions client.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ type Client struct {
// Dial connects to Temporal and starts OTLP log export when it is configured; see
// InitLogExport.
//
// An empty HostPort falls back to TEMPORAL_ENDPOINT, then TEMPORAL_ADDRESS, then the
// SDK's default of localhost:7233. An empty Namespace falls back to TEMPORAL_NAMESPACE,
// then the SDK's default namespace.
// An empty HostPort falls back to TEMPORAL_ADDRESS, then the SDK's default of
// localhost:7233. An empty Namespace falls back to TEMPORAL_NAMESPACE, then the SDK's
// default namespace.
func Dial(options Options) (*Client, error) {
if options.HostPort == "" {
options.HostPort = temporalEndpointFromEnv()
options.HostPort = os.Getenv(envTemporalAddress)
}
if options.Namespace == "" {
options.Namespace = os.Getenv(envTemporalNamespace)
Expand Down
37 changes: 12 additions & 25 deletions config_json_test.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package gush_test

import (
"encoding/json"
"testing"
"time"

Expand All @@ -12,7 +11,6 @@ func TestFlowConfigMarshalUsesSnakeCaseJSONTags(t *testing.T) {
config := gush.DefaultFlowConfig()
config.ID = "flow-123"
config.Timeout = 5 * time.Minute
config.Cron = "0 2 * * *"
config.Priority = gush.PriorityHigh
config.FairKey = "tenant-a"
config.InitialInterval = 2 * time.Second
Expand All @@ -36,14 +34,13 @@ func TestFlowConfigMarshalUsesSnakeCaseJSONTags(t *testing.T) {
expectedKeys := []string{
"id",
"timeout",
"cron",
"priority",
"fairness",
"fair_key",
"initial_interval",
"backoff_coefficient",
"maximum_interval",
"maximum_attempts",
"non_retryable_error_types",
"max_interval",
"max_attempts",
"no_retry",
"log_attributes",
}

Expand All @@ -54,18 +51,6 @@ func TestFlowConfigMarshalUsesSnakeCaseJSONTags(t *testing.T) {
}
}

// A config can wait in a queue, so it may hold keys its reader no longer has.
func TestFlowConfigIgnoresKeysItDoesNotKnow(t *testing.T) {
var config gush.FlowConfig
err := json.Unmarshal([]byte(`{"id":"flow-1","async":true}`), &config)
if err != nil {
t.Fatalf("Unmarshal returned error: %v", err)
}
if config.ID != "flow-1" {
t.Errorf("ID = %q", config.ID)
}
}

func TestStepConfigMarshalUsesSnakeCaseJSONTags(t *testing.T) {
config := gush.DefaultStepConfig()
config.Queue = "queue-a"
Expand Down Expand Up @@ -93,15 +78,17 @@ func TestStepConfigMarshalUsesSnakeCaseJSONTags(t *testing.T) {

expectedKeys := []string{
"queue",
"timeout",
"worker_pickup_timeout",
"attempt_timeout",
"pickup_timeout",
"heartbeat_timeout",
"wait_for_cancel",
"priority",
"fairness",
"fair_key",
"initial_interval",
"backoff_coefficient",
"maximum_interval",
"maximum_attempts",
"non_retryable_error_types",
"max_interval",
"max_attempts",
"no_retry",
"log_attributes",
}

Expand Down
8 changes: 0 additions & 8 deletions env.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,9 @@ package gush
import "os"

const (
envTemporalEndpoint = "TEMPORAL_ENDPOINT"
envTemporalAddress = "TEMPORAL_ADDRESS"
envTemporalNamespace = "TEMPORAL_NAMESPACE"

envLogGRPCEndpoint = "OTEL_ENDPOINT"
envLogHTTPEndpoint = "OTEL_HTTP_ENDPOINT"

envOTLPLogsEndpoint = "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT"
envOTLPEndpoint = "OTEL_EXPORTER_OTLP_ENDPOINT"
envOTLPLogsProtocol = "OTEL_EXPORTER_OTLP_LOGS_PROTOCOL"
Expand All @@ -26,7 +22,3 @@ func firstEnv(names ...string) string {
}
return ""
}

func temporalEndpointFromEnv() string {
return firstEnv(envTemporalEndpoint, envTemporalAddress)
}
11 changes: 1 addition & 10 deletions flow.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,11 @@ package gush

import (
"context"
"errors"
"time"

"go.temporal.io/sdk/client"
)

// errCronNeedsStartFlow is what RunFlow returns for a schedule, which never gives it a
// result to wait for.
var errCronNeedsStartFlow = errors.New("gush: a cron schedule has no result for RunFlow to wait for; use StartFlow")

// FlowHandlerFunc is the signature of a flow.
type FlowHandlerFunc func(ctx *FlowContext) (output any, err error)

Expand Down Expand Up @@ -64,14 +59,10 @@ type FlowInfo struct {
CloseTime *time.Time // nil while running
}

// RunFlow starts a flow and waits for its result. It returns an error for a WithCron
// schedule, which never gives a result to wait for, so start one with StartFlow.
// RunFlow starts a flow and waits for its result.
func (c *Client) RunFlow(ctx context.Context, request FlowRequest, options ...FlowOption) error {
config := DefaultFlowConfig()
config.Apply(options...)
if config.Cron != "" {
return errCronNeedsStartFlow
}

run, err := c.startFlow(ctx, request, config)
if err != nil {
Expand Down
31 changes: 5 additions & 26 deletions flow_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ import (

// FlowConfig holds the settings of one flow run. Its JSON form is stable, so a config can
// be queued and applied later with WithConfig. A field left out of the JSON keeps its
// default, while a zero written out is taken as it is, so "maximum_attempts": 0 still
// default, while a zero written out is taken as it is, so "max_attempts": 0 still
// means unlimited. Waiting is not part of a config: whoever applies it chooses RunFlow or
// StartFlow, and one with a Cron needs StartFlow.
// StartFlow.
//
// Like time.Time, it decodes itself from JSON, so a struct that embeds it would decode
// through that method and lose its own fields. Hold it in a named field to decode it.
Expand All @@ -23,14 +23,8 @@ type FlowConfig struct {
// ID is the flow ID. If empty, one is generated.
ID string `json:"id"`

// Timeout limits the whole execution, including every retry and every scheduled run.
// Zero means unlimited.
// Timeout limits the whole execution, including every retry. Zero means unlimited.
Timeout time.Duration `json:"timeout"`

// Cron is a schedule in UTC, such as "0 2 * * *" for daily at 2 AM. A run starts only
// after the previous one ends, and the schedule continues until canceled, terminated
// or stopped by Timeout.
Cron string `json:"cron"`
}

// DefaultFlowConfig returns the settings a flow run starts from: one attempt, medium
Expand Down Expand Up @@ -111,33 +105,18 @@ func WithConfig(config *FlowConfig) FlowOption {

// WithID sets the flow ID. Without it one is generated, which StartFlow returns. While a
// flow with this ID is running, a client's RunFlow or StartFlow joins it instead of starting
// another and ignores this run's input, and a child flow gets an error. RunFlow then waits
// for that flow, even across a WithCron schedule.
// another and ignores this run's input, and a child flow gets an error.
func WithID(flowID string) FlowOption {
apply := func(config *FlowConfig) {
config.ID = flowID
}
return flowOption(apply)
}

// WithTimeout limits the whole flow execution, including every retry and every scheduled
// run of a WithCron schedule, so it also ends the schedule.
// WithTimeout limits the whole flow execution, including every retry.
func WithTimeout(timeout time.Duration) FlowOption {
apply := func(config *FlowConfig) {
config.Timeout = timeout
}
return flowOption(apply)
}

// WithCron runs the flow on a schedule in UTC, such as "0 2 * * *" for daily at 2 AM. Start
// it with StartFlow, because RunFlow returns an error: a schedule has no result to wait for.
//
// A client's StartFlow returns an error for an invalid expression. On a child flow, Temporal
// instead fails the parent's workflow task, which retries until the parent's WithTimeout, so
// check an expression that comes from outside first.
func WithCron(cron string) FlowOption {
apply := func(config *FlowConfig) {
config.Cron = cron
}
return flowOption(apply)
}
48 changes: 5 additions & 43 deletions flow_config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,6 @@ func TestDefaultFlowConfig(t *testing.T) {
if config.ID != "" {
t.Errorf("Expected ID to be empty, got %v", config.ID)
}
if config.Cron != "" {
t.Errorf("Expected Cron to be empty, got %v", config.Cron)
}

// Test LogAttributes initialization
if config.LogAttributes == nil {
Expand Down Expand Up @@ -81,13 +78,6 @@ func TestFlowOptions(t *testing.T) {
return c.Priority == gush.PriorityHigh
},
},
{
name: "WithCron",
option: gush.WithCron("0 2 * * *"),
expected: func(c *gush.FlowConfig) bool {
return c.Cron == "0 2 * * *"
},
},
{
name: "WithRetryPolicy",
option: gush.WithRetryPolicy(retryPolicy),
Expand Down Expand Up @@ -119,7 +109,6 @@ func TestMultipleFlowOptions(t *testing.T) {
gush.WithID("batch-process-123"),
gush.WithTimeout(10 * time.Minute),
gush.WithPriority(gush.PriorityLow),
gush.WithCron("0 3 * * *"),
gush.WithLogAttribute("user_id", "batch-user"),
gush.WithLogAttribute("process_type", "batch"),
}
Expand All @@ -138,9 +127,6 @@ func TestMultipleFlowOptions(t *testing.T) {
if config.Priority != gush.PriorityLow {
t.Errorf("Expected Priority to be PriorityLow, got %v", config.Priority)
}
if config.Cron != "0 3 * * *" {
t.Errorf("Expected Cron to be '0 3 * * *', got %v", config.Cron)
}
if config.LogAttributes["user_id"] != "batch-user" {
t.Errorf("Expected LogAttributes[user_id] to be 'batch-user', got %v", config.LogAttributes["user_id"])
}
Expand Down Expand Up @@ -183,7 +169,6 @@ func TestWithConfigAllowsLaterOverrides(t *testing.T) {
},
ID: "config-flow-id",
Timeout: 15 * time.Minute,
Cron: "0 4 * * *",
}

options := []gush.FlowOption{
Expand All @@ -203,9 +188,6 @@ func TestWithConfigAllowsLaterOverrides(t *testing.T) {
if config.Timeout != 20*time.Minute {
t.Errorf("Expected Timeout to be overridden, got %v", config.Timeout)
}
if config.Cron != "0 4 * * *" {
t.Errorf("Expected Cron to come from custom config, got %v", config.Cron)
}
if config.Priority != gush.PriorityHigh {
t.Errorf("Expected Priority to come from custom config, got %v", config.Priority)
}
Expand Down Expand Up @@ -250,26 +232,6 @@ func TestFlowConfigRetryPolicyInheritance(t *testing.T) {
}
}

func TestCronScheduleValidation(t *testing.T) {
validCronSchedules := []string{
"0 2 * * *", // Daily at 2 AM
"0 */6 * * *", // Every 6 hours
"0 9 * * 1", // Every Monday at 9 AM
"30 14 * * *", // Daily at 2:30 PM
"0 0 1 * *", // First day of every month
}

for _, cron := range validCronSchedules {
t.Run("ValidCron_"+cron, func(t *testing.T) {
config := gush.DefaultFlowConfig()
config.Apply(gush.WithCron(cron))
if config.Cron != cron {
t.Errorf("Expected Cron to be %s, got %s", cron, config.Cron)
}
})
}
}

func decodedConfig(t *testing.T, raw string) *gush.FlowConfig {
t.Helper()

Expand Down Expand Up @@ -297,7 +259,7 @@ func TestPartialJSONConfigKeepsTheDefaults(t *testing.T) {
}

func TestJSONConfigTakesAWrittenZero(t *testing.T) {
config := decodedConfig(t, `{"maximum_attempts":0}`)
config := decodedConfig(t, `{"max_attempts":0}`)

if config.MaxAttempts != 0 {
t.Errorf("attempts = %d, want the written 0, which means unlimited", config.MaxAttempts)
Expand Down Expand Up @@ -358,7 +320,7 @@ func TestWithConfigOverridesWhatItSets(t *testing.T) {
inherited := gush.DefaultFlowConfig()
inherited.Priority = gush.PriorityHigh

config := decodedConfig(t, `{"priority":"low","fairness":"tenant-2"}`)
config := decodedConfig(t, `{"priority":"low","fair_key":"tenant-2"}`)
inherited.Apply(gush.WithConfig(config))

if inherited.Priority != gush.PriorityLow || inherited.FairKey != "tenant-2" {
Expand All @@ -372,13 +334,13 @@ func TestJSONConfigDecodesOverExistingSettings(t *testing.T) {
config.MaxAttempts = 5
config.Priority = gush.PriorityHigh

err := gush.Decode([]byte(`{"cron":"0 2 * * *"}`), config)
err := gush.Decode([]byte(`{"timeout":60000000000}`), config)
if err != nil {
t.Fatalf("Decode() error = %v", err)
}

if config.Cron != "0 2 * * *" {
t.Errorf("cron = %q", config.Cron)
if config.Timeout != time.Minute {
t.Errorf("timeout = %v", config.Timeout)
}
if config.ID != "kept" || config.MaxAttempts != 5 || config.Priority != gush.PriorityHigh {
t.Errorf("existing settings changed: %q, %d, %q", config.ID, config.MaxAttempts, config.Priority)
Expand Down
Loading