docs: Tango Errors RFC

[justinwon777]

Adds a Tango Errors RFC covering the core/errors package and its APIs

[sbalabanov]

Few things are missing in the RFC design:

• Clear error domain definition. What types of errors could ever be in the system? (see comments on data structures).
• Guidelines on using classifiers; where exactly should classification be done and what data will it use. In the provided example, bazel query error may be both infra (i.e. no bazel binary on the machine) or user (i.e. change is incompatible with the trunk). Infra might be retryable (i.e. bazilisk failed with http error) or not (i.e. incorrect argument list to this specific bazel version).
• How internal classification translates to the API layer (GRPC codes and rich error object). GRPC defines its own codes which cannot be changed or extended, typically observable by common infrastructure (i.e. muttley).
• Missing dependency flag. We would like to compute inclusive and exclusive reliability metric for the service, i.e. it should differentiate between its own errors (tango process + subprocesses it runs) and external systems it calls (i.e. CI or storage).

[sbalabanov]

we should be able to differentiate between missing arg (which is a programming error on a caller side) vs build failure (which is a user error not related to the ecosystem correctness)

[justinwon777]

I agree we should eventually have different error codes for different types of user failures like canceled and build failure, but I think we should take advantage of that once our error classification is more mature. For instance, right now there is no way for Tango to know why a buildkite build failed, so there is no use of having a build failure error code.

[justinwon777]

I added a cancelled error code since we see that one a lot and think it'd be helpful to distinguish cancelled from other user failures.

[sbalabanov]

what is a difference between UNKNOWN and INTERNAL?

[justinwon777]

UNKNOWN is used as a default if an error is never classified. We have a lot of places currently returning plain errors, so they will show up as unknown, and we can go through them and classify them. Sort of like how we use generic in our buildkite classifier.

[sbalabanov]

in the example above, String() is exported

[justinwon777]

There's an additional unexported method not shown here.

[justinwon777]

@sbalabanov partial response, still looking into your GRPC codes comment

• Guidelines on using classifiers; where exactly should classification be done and what data will it use. In the provided example, bazel query error may be both infra (i.e. no bazel binary on the machine) or user (i.e. change is incompatible with the trunk). Infra might be retryable (i.e. bazilisk failed with http error) or not (i.e. incorrect argument list to this specific bazel version).

I added a more detailed example of the usage. This package's purpose is to provide the framework for callers to classify an error, and the caller is responsible for determining how to classify an error. At the controller layer, the mapper will handle all errors and convert them to the proto type (unclassified will be unknown error code). Some packages like bazel will need more work done to differentiate user and infra failures.

• Missing dependency flag. We would like to compute inclusive and exclusive reliability metric for the service, i.e. it should differentiate between its own errors (tango process + subprocesses it runs) and external systems it calls (i.e. CI or storage).

Yes we could potentially add two more error codes ErrorInternalDependency and ErrorInternalDependencyRetryable. In a future task we can identify external systems that can use these and then add the error codes.

[justinwon777]

• How internal classification translates to the API layer (GRPC codes and rich error object). GRPC defines its own codes which cannot be changed or extended, typically observable by common infrastructure (i.e. muttley).

Added the gRPC code mapping to the error code definitions