[LIVY-552][WIP] Add JWTFilter to validate incoming JWT tokens

[z-york]

What changes were proposed in this pull request?

This PR adds JWT Authentication to Livy via a new filter, JWTFilter that validates the JWT and accepts or rejects the request.

https://issues.apache.org/jira/browse/LIVY-552

(Please fill in changes proposed in this fix)
(Include a link to the associated JIRA and make sure to add a link to this pr on the JIRA as well)

How was this patch tested?

Unit Tests passed.
Tested manually as well.

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

Please review https://livy.incubator.apache.org/community/ before opening a pull request.

[codecov-io]

Codecov Report

Merging #142 into master will increase coverage by 0.28%.
The diff coverage is 71.87%.

@@             Coverage Diff              @@
##             master     #142      +/-   ##
============================================
+ Coverage     68.34%   68.62%   +0.28%     
- Complexity      895      913      +18     
============================================
  Files           100      103       +3     
  Lines          5604     5664      +60     
  Branches        840      848       +8     
============================================
+ Hits           3830     3887      +57     
+ Misses         1225     1220       -5     
- Partials        549      557       +8

Impacted Files	Coverage Δ	Complexity Δ
...main/scala/org/apache/livy/server/LivyServer.scala	37.11% <0%> (-0.59%)	11 <0> (ø)
...rver/src/main/scala/org/apache/livy/LivyConf.scala	95.91% <100%> (+0.06%)	21 <0> (ø)	⬇️
...n/scala/org/apache/livy/server/jwt/JWTFilter.scala	66.66% <66.66%> (ø)	6 <6> (?)
...cala/org/apache/livy/server/jwt/JWTValidator.scala	85.71% <85.71%> (ø)	9 <9> (?)
...g/apache/livy/server/jwt/JWSVerifierProvider.scala	93.75% <93.75%> (ø)	5 <5> (?)
...ain/java/org/apache/livy/rsc/driver/RSCDriver.java	77.96% <0%> (-1.28%)	41% <0%> (-1%)
...main/java/org/apache/livy/rsc/ContextLauncher.java	66.01% <0%> (+1.94%)	13% <0%> (ø)	⬇️
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02550f7...101a6e2. Read the comment docs.

[alex-the-man]

Is there any chances we need to support multiple public keys for key rotation?

[z-york]

Do you mean allowing users to specify a list of public keys to workaround having to restart livy when a particular key is rotated? I think it's an interesting idea, but would require other changes to work (currently the key is read in once from the path and not refreshed, this would require some refresh mechanism). Anyways, I think this can be a separate feature request/JIRA after this one gets in.

[alex-the-man]

LIVY-508 #110 will be merged soon and will provide a framework to add custom filters.
Would you mind rebasing your work on top of that? The only required changes should be all in LivyConf.

# If user wants to use custom authentication filter, configurations are:
# livy.server.auth.type = <custom>
# livy.server.auth.<custom>.class = <class of custom auth filter>
# livy.server.auth.<custom>.param.<foo1> = <bar1>
# livy.server.auth.<custom>.param.<foo2> = <bar2>

[z-york]

Sure, let me try to rebase ontop of that PR

[Tagar]

@z-york btw #110 is now merged to master.

[z-york]

Thanks for the heads up @Tagar! @alex-the-man I took a look at rebasing it ontop (and using) 110 and I think there are a few things that don't make sense with the current approach:

• It looks like only one auth_type can be specified which has a number of problems:
  ** Filters should be composable
  ** You should be able to use a custom filter/auth type with Kerberos

In addition to this, the JWTFilter in my PR requires LivyConf to be passed in when it is created which I assume won't work if it is being instantiated via the Filter Holder (not sure how this is being done, but I would assume reflection).

I think it should be fairly simple to change the config to a list of auth types which will solve the first problem, but I'm not sure about how to solve the 2nd issue. Any ideas?

[alex-the-man]

@jerryshao What is your opinion on supporting multiple filters?

[alex-the-man]

I think it should be fairly simple to change the config to a list of auth types which will solve the first problem, but I'm not sure about how to solve the 2nd issue. Any ideas?

Those config will be passed to the function:

override def init(filterConfig: FilterConfig): Unit = {}

Could you change JWTFilter to pass those config to JWSVerifierProvider in that function call?

[z-york]

My apologies for such a long lapse here... I will be picking this back up again.

[naksh9619]

@z-york Are you planning to continue working on this? Otherwise I can pick up from where you left

[z-york]

@naksh9619 Feel free to continue the work. I have since been pulled onto other work and didn't have the time to address follow-up comments.

[github-actions]

This pull request has been automatically marked as stale because it has had no activity for at least 3 months. If you are still working on this change or plan to move it forward, please leave a comment or push a new commit so we know to keep it open. Otherwise, this PR will be closed automatically in about one month. Thank you for your contribution to Apache Livy!