Conversation
keith-turner
left a comment
There was a problem hiding this comment.
Would be nice to add an IT for this. Could look at the ITs that use the class TestStatsDSink for examples of how to create an IT that reads metrics.
Looked around for an IT that exercises scan executors and did not see one, its a purely performance feature and hard to test. Can look at these docs for info on setting up scan executors. Need to do something like the following in the IT.
- Before the cluster starts set some
tserver.scan.executors.<executor name>.threads=<num>props in the site config to create different named scan executors. - Create a few tables and set them to have different scan executors by setting the
table.scan.dispatcher.opts.executor=<executor name>props. - Run a scan against a table w/ an iterator that will throw an exception. Check the metrics for errors for the executor the table was configured to use.
In a way this test will also help us verify that scan dispatching is working correctly and sending things to the expected executor on the server side.
core/src/main/java/org/apache/accumulo/core/metrics/MetricsProducer.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/apache/accumulo/core/metrics/MetricsProducer.java
Show resolved
Hide resolved
server/tserver/src/main/java/org/apache/accumulo/tserver/scan/NextBatchTask.java
Outdated
Show resolved
Hide resolved
server/tserver/src/main/java/org/apache/accumulo/tserver/scan/NextBatchTask.java
Show resolved
Hide resolved
|
Not a change for this PR, but its interesting how this PR is tagging a metric w/ the scan executor name. Could potentially tag all scan metrics w/ the executor name. |
|
Be sure to run the MetricsIT and other Metrics-related ITs. |
test/src/main/java/org/apache/accumulo/test/metrics/ScanExecutorExceptionsIT.java
Outdated
Show resolved
Hide resolved
test/src/main/java/org/apache/accumulo/test/metrics/ScanExecutorExceptionsIT.java
Outdated
Show resolved
Hide resolved
dlmarion
left a comment
There was a problem hiding this comment.
I think this is looking better. A few minor comments.
Closes issue #5057
Introduced a new metric to track the number of exceptions that occur within the scan executor. Allows for users to monitor exceptions thrown from the iterator stack, instead of relying on parsing logs.