[LIVY-508][Server] Support custom auth filter for livy server

[jerryshao]

What changes were proposed in this pull request?

Users may have the requirement to add their own auth filter to the Livy server other than Kerberos, so here propose to add this support.

How was this patch tested?

Verified locally in the cluster.

[jerryshao]

@zjffdu @ajbozarth would you please help to review, thanks!

[zjffdu]

It would be better to have document for this. e.g. how to create custom auth filter, and how to deploy it.

[ajbozarth]

I agree with @zjffdu that documentation would be good to have, either in the docs or on the website, but from a code side this looks concise and solid. I have no issues, LGTM

[vanzin]

To be fair there isn't really any documentation about configuring the server. The config template file is currently the most complete source of information for that, so I'm fine with just the example that was added there.

[vanzin]

Use a more descriptive name (e.g. customType)?

[alex-the-man]

Can I merge this? I believe this will enable #142.

[vanzin]

I think it's fine to merge, I just had a nit comment.

[jerryshao]

Sorry for the delay, I will address the comments soon.

[codecov-io]

Codecov Report

Merging #110 into master will decrease coverage by 0.19%.
The diff coverage is 0%.

@@             Coverage Diff             @@
##             master     #110     +/-   ##
===========================================
- Coverage     68.54%   68.34%   -0.2%     
+ Complexity      893      891      -2     
===========================================
  Files           100      100             
  Lines          5604     5616     +12     
  Branches        840      842      +2     
===========================================
- Hits           3841     3838      -3     
- Misses         1212     1225     +13     
- Partials        551      553      +2

Impacted Files	Coverage Δ	Complexity Δ
...main/scala/org/apache/livy/server/LivyServer.scala	35.46% <0%> (-2.23%)	11 <0> (ø)
...cala/org/apache/livy/scalaapi/ScalaJobHandle.scala	52.94% <0%> (-2.95%)	7% <0%> (ø)
...ain/java/org/apache/livy/rsc/driver/RSCDriver.java	76.69% <0%> (-2.12%)	39% <0%> (-2%)
...ain/scala/org/apache/livy/utils/SparkYarnApp.scala	73.23% <0%> (+2.11%)	33% <0%> (ø)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5489d0...7f6124f. Read the comment docs.

[mgaido91]

LGTM too, merging to master. Thanks.

[johnnydepup]

To be fair there isn't really any documentation about configuring the server. The config template file is currently the most complete source of information for that, so I'm fine with just the example that was added there.

I know it's an old thread, but the given example configuration in the file is not very helpful. It doesn't mention, for example, which class the Filter should even extend. It hasn't been updated since then even after two years.

I think a proper example with some sample code would be really helpful for new users.

I still don't understand what the params are for and how to use them. An example describing all of these would be quite useful.

[johnnydepup]

And with the following configuration, livy UI doesn't give me any option to enter a username or password.

livy.server.auth.type = dis
livy.server.auth.dis.class = com.dis.DISAuthenticationFilter
livy.server.auth.dis.param.username = user
livy.server.auth.dis.param.password = password

What other parameters need to be passed?