Out of memory when too much assets

[alkollo]

I got some memory problem when I use more than 10 assets to rebalance.

My server is 64 GB RAM which are completly used and lead to a "process stopped", after swap was also fully used.

I suspect the loop "while todo" in the method balancer from simple_balancer.py. Is it a know "problem" ?

Is it a normal behavior and number of assets is limited to RAM available ?

Best regards,

[m1cedricvanheck]

Hi Alkollo,

Any updates on this issue? I'm dealing with the same problem and would love to know if there is any solution to it. Did not found one myself yet.

Best regards

[alkollo]

Not for now, I just used a server with 128 GB RAM, I'm wondering if changing SWAP to let's says 500 MB could temporally solve the problem...

Also lowering to few assets fix the problem but this is not the best to deal with in my opinion.

Hope the author could solve this problem someday.

Cheers,

[m1cedricvanheck]

Thanks for the quick reply. I solved the problem just recently by ordering and picking the best attempt after x iterations and then start clean again. At the end, it chooses the best attempt out of this list with best attempts. As a result, It only uses a fixed amount of memory. In case you are interested, I can share with you the piece of code. The only remaining issue is that it takes a lot of time in order to create the best set of orders. I’m using almost 30 assets and it takes hours to optimise. Did you encounter this problem as well, and if so, how did you deal with this? Cheers! From: alkollo ***@***.***> Sent: maandag 9 mei 2022 16:12 To: hammertoe/crypto_balancer ***@***.***> Cc: Cedric van Heck ***@***.***>; Comment ***@***.***> Subject: Re: [hammertoe/crypto_balancer] Out of memory when too much assets (#14) Not for now, I just used a server with 128 GB RAM, I'm wondering if changing SWAP to let's says 500 MB could temporally solve the problem... Also lowering to few assets fix the problem but this is not the best to deal with in my opinion. Hope the author could solve this problem someday. Cheers, — Reply to this email directly, view it on GitHub<#14 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AZCWBZEHUUDHJPW2ND4IA7DVJEMK7ANCNFSM46TNHBZQ>. You are receiving this because you commented.Message ID: ***@***.******@***.***>>

[alkollo]

Would love to have that piece of code, I was not able to figure it out :)

For the hours of computing it is not a problem as I only do daily rebalancing, I have read an article where it says daily is the most efficient interval (I think on Shrimpy blog) so I did not dig further. By the way it needs a lot of computing when it have to rebalance, once the balancing is stable (not too much assets to rebalance because of low fluctuation) it compute faster I noticed. 30 assets seems a lot to me, just my 2cts.

Regards,

[draczer01]

any updates? I created a fork of this repo and I'm currently adding new features, like the possibility to use a proxy and an additional library so it can be called and used by another Python program without requiring the config.ini file, I'm also working on keeping the libraries up to date. @m1cedricvanheck I would like to see your fix so I can implement it on my fork pls
here is the fork:
https://github.com/draczer01/Crypto_Reflex

[m1cedricvanheck]

simple_balancer.zip

Here it is. I guess you have to tweak it a little bit since I haven't used in for a while now. I have included some comments. At least it keeps running (also with a lot of assets) but the problem is that it is not converging to a optimal result. At least not quick enough. Succes with it!

[draczer01]

Thanks, ill tweak and work with it to achieve something optimal or at least closer to optimal

[cbh4ou]

Will be forking, and start development on this. I've written my own private rebalancer, but its gotten pretty messy. No issues with tons of assets, so possibly I can blend in the solution. We'll see, I'll keep you updated.

[m1cedricvanheck]

Sounds interesting. Looking forward to the end result. Keep us updated please! Thanks

[cbh4ou]

Mine took a similar iterative approach in real time, rather than pre-generating orders that could fail. I keep trying the best as possible and optimize if it fails. I dont like failing orders where we know we can close them, especially with algorithm that will be on for long periods of time. The smart pair routing will be the most difficult part, but I think building a trade instruction with steps will be the best way, then we optimize the order sizes until its accepted by the exchange (unless impossible to close. ex: trade mins, not enough balance.) The permutations aren't enough, but it is a start to build on.