Proposal: make the editor bit of play.haskell.org use a haskell-language-server instance in the background

[MangoIV]

Hello Tom
I am currently working on getting HLS to work with fully in-memory buffers that do not need the file system to be able to have a single HLS instance communicate with multiple language server clients. I have reached the point where I have all relevant language server features working in a prototype-ish way and I would like to avoid scattering the ecosystem by creating another playground. Hence I would like to ask whether what I have built could be integrated into haskell playground.

• the language server would for now be a fork since I don't think that most of the feature will be mergeable upstream immediately
• a single language server instance takes about 1.5 GB RAM for ~30 active clients
• we probably need proper sandboxing, I don't know how haskell playground does this at the moment, but we probably need to talk about this
• the client is based on monaco, which is pretty big, so the download times would probably go up.

If you have any further questions, I would be happy to talk.

Here's a screenshot of how this looks atm - the missing syntax highlighting is due to not having any except the semantic highlighting atm. I could add syntax based highlighting client side but I'm also currently debating on just extending the range of supported tokens by HLS.

My e-mail address is contact@mangoiv.com

[MangoIV]

Also if that wasn't clear - I am of course happy to implement the proposed change myself. The typescript code is very simple and you basically just end up mounting the monaco editor into an empty div.

[tomsmeding]

This is pretty cool! I've never considered providing HLS on the haskell playground because of the excessive server-side memory requirements of running a single HLS instance per client. But it seems your fork makes this a lot better. What I do wonder about: you mention sandboxing; are the GHC sessions of the various clients isolated from each other inside HLS? What happens if one of the buffers runs random IO stuff using TemplateHaskell, or declares a GHC plugin (if that's possible using the packages currently available on the playground)?

Integrating this into the current technical design of the playground would be a bit of work, but far from impossible. The tricky bit is that the setup is a little idiosyncratic, and I currently have very little time to spend on this. I could do some asynchronous mentoring, though.

Thinking ahead to resource requirements on the server: this would at least need the option to reject a client's HLS session if memory usage is already too high, and terminate sessions when necessary. Depending on usage, we might need a larger server, but then funding becomes something to think about.

[MangoIV]

are the GHC sessions of the various clients isolated from each other inside HLS? What happens if one of the buffers runs random IO stuff using TemplateHaskell, or declares a GHC plugin (if that's possible using the packages currently available on the playground)?

yeah, this is part of my concerns wrt sandboxing, I don't have clear answers on these questions yet.

I could do some asynchronous mentoring, though.

that would be really nice and I think I wouldn't need much more from you anyway - my main concern is that you're interested in this at all which you are, which makes me really happy.

Thinking ahead to resource requirements on the server: this would at least need the option to reject a client's HLS session if memory usage is already too high, and terminate sessions when necessary. Depending on usage, we might need a larger server, but then funding becomes something to think about.

This is true. It's hard to wager atm how big of a machine you would need in the end and who would be able to run it - do you have any information for me on how the setup works atm? i.e. on what kinda machine play.haskell.org runs, who owns it, what kinda resource usage it typically sees?

Thank you so much ~

[tomsmeding]

do you have any information for me on how the setup works atm? i.e. on what kinda machine play.haskell.org runs, who owns it, what kinda resource usage it typically sees?

It's running on a Hetzner CPX21 server (see here under "shared vCPU AMD"), server and worker on the same machine. I "own" (to the extent that you own a virtual server) the thing and pay for it. Setup: some slides. Usage: graph, every red pixel is a job request (run/core/asm), every green pixel is a second of handling time spent on the worker; both count from the bottom of the image, green overlaps red. I don't have good stats on CPU/RAM usage, but anecdotally the server handles the load no problem with room to spare, especially CPU-wise.

are the GHC sessions of the various clients isolated from each other inside HLS? What happens if one of the buffers runs random IO stuff using TemplateHaskell, or declares a GHC plugin (if that's possible using the packages currently available on the playground)?

yeah, this is part of my concerns wrt sandboxing, I don't have clear answers on these questions yet.

I didn't think about this when first replying to your post here, but I think this is an important question to answer. I would like the experience of user 1 to be independent (perhaps apart from slowness) of whatever other users are doing on the playground. Messing with HLS' internal state would perhaps not impact the run results on the playground, which is a good thing, but I still don't like it much.

One may think of using SafeHaskell to prohibit TemplateHaskell, and LambdaBot on IRC does rely on that, but I don't trust it much: GHC regularly has bugs (like any big open source project apart from perhaps CompCert; this is not a criticism of GHC) and the whole reason sandboxing is a thing that we do here is because bugs in software exist, and we'd like to ensure isolation even in the face of them. Or as much as we can, because sandboxing software is also software.
Besides, SafeHaskell does not seem to prohibit the use of GHC plugins.