1677 return stack trace for authenticated users with specific roles

[krowvin]

Add conditional stack traces for local and development debugging and keeps the existing incidentIdentifier behavior used for log discovery.

Stack traces are now included in CdaError.details.stackTrace when the request is
explicitly configured for trace output:

• request is on localhost (docker compose)
  OR
• request is authenticated as a CWMS User Admins AND in dev environment (env.lower().includes('dev'))

The implementation keeps CdaError as a response DTO and places the request-aware policy
in ErrorTraceSupport.

To make that behavior apply consistently, this PR also removes several redundant
controller-level catch (Exception) wrappers that were bypassing ApiServlet, and
updates remaining endpoint 500 handlers to make their responses through
ErrorTraceSupport.

Partly fixes some of

• Improve exception handling #1625
  but not the ConnectionPreparingDataSource

Other potentially related issues include:

• Improve Exception/Error handling logic, heirarchy, etc #1106
• Be more principled about Exception handling #299
• JooqDao changes removed exception wrapping #350
• Cannot retrieve Elev Timeseries #1342 (helps with diagnosing/better traceback visibility)

Trying to centralize general errors where possible and reduce fragmentation

Not trying to solve a new base exception direction, or a better inheritance model mainly because @MikeNeilson said to discuss more before doing that in another issue.

[krowvin]

Something to note is that it returns JSON and a stack trace in JSON is not the prettiest of things with all the \n in the JSON string.

The last commit shows an alternative where we could keep both, or pick one, of the options. Putting the traces in a list split by the newline (Could probably do \n\t too) does render nicely in the browser at least and possible in a fetch/output of a downstream client.

List

Vs

String

[MikeNeilson]

Instead of using System.getProperty or System.getenv directly, integrate into the Togglz feature system.

But otherwise I don't hate it. Do want to chew on it some more, but overall seems reasonable.

[krowvin]

Spoke with Mike. Lets go with stackTraceLines and remove the string.

Need to fix the OpenAPI spec. Latest Schema can break if Oracle out of sync (but check). But search for FAILED all caps and first instance all caps FAILED should have your error for the other failed tests.

[MikeNeilson]

Looks good. However, there is one challenge you need to do.

An integration test that actually renders a stacktrace. NOTE: you may need to be a bit contrived here, except that reality.

[krowvin]

Removed these so the tests would pass

[MikeNeilson]

Generally looks good. A few nitpicks about some of the tests.

[MikeNeilson]

This test should be of the actual API returning the stack trace appropriate.

There should be no need to Proxy HTTP response or Request. This should be derived from DataApiTestIT so that it runs after the CDA instance is setup.

However, I will grant that it may be a bit difficult to force something to actually output a stack trace. You should be able insert a new handler, within the test code, that does that.

[MikeNeilson]

Move this to it's own class under fixtures.