Thoughts on Model Management

[fiatrete]

After play with Playground for a while, I feel that the models supported by DAN are still too few. I am thinking about how to make DAN support as many models as possible.

Here is a possible solution:

• dan-server provides a list of supported models.
• Donors decide which models to add to their donated nodes.
• The model list on the node is automatically reported to the dan-server.
• API users specify the model in the task.
  • After receiving the task, dan-server searches for nodes that have these models.
  • Among the nodes that have the target model, the one with the lowest load is selected to schedule the task.

This solution requires dan-node to provide a set of management functions, so the implementation of this solution may have to be postponed until the implementation of custom nodes.

[sharpbai]

Hmm,

• Dornors decide which models to add or not seems make the donation process complicated.
• the flexiblity of model selection on donated nodes leads to n^2 level increased complexity in scheduling which request should send to which node, which is not suitable for single-point dan-server
• the flexibility of model selection on donated nodes leads to unexpected generation power of different models, which makes some generation requests never or hardly be completed, due to the difference between donor expectations and producer needs

Make the model selection less flexible while keep the multi-model support capability seems important in design.

[fiatrete]

Hmm,

• Dornors decide which models to add or not seems make the donation process complicated.

the approach will not make the donation process complicated, but will require donors to put more effort into managing donated nodes.

• the flexiblity of model selection on donated nodes leads to n^2 level increased complexity in scheduling which request should send to which node, which is not suitable for single-point dan-server

by setting up an index of <model, node> mapping, the operation can be completed in O(1) complexity.

• the flexibility of model selection on donated nodes leads to unexpected generation power of different models, which makes some generation requests never or hardly be completed, due to the difference between donor expectations and producer needs

This is inevitable. We need a solution that classifies the workload of tasks and the computing power of nodes. However, that's another story. We can add it as a to-do item to our roadmap for now.

Make the model selection less flexible while keep the multi-model support capability seems important in design.

agree

[Ox2098]

Your model is very poor

[suntodai]

Similar to many ML frameworks such as tensorflow, pytorch etc. which provide various models in a model zoo, we could add a few (e.g. 3-5) typical/SOTA models with wide applications. With more contributors later on, more models/APIs could be integrated.

[fiatrete]

Similar to many ML frameworks such as tensorflow, pytorch etc. which provide various models in a model zoo, we could add a few (e.g. 3-5) typical/SOTA models with wide applications. With more contributors later on, more models/APIs could be integrated.

Absolutely! As an open AI infrastructure platform, it's crucial for model support to be freely available. However, we can take a step-by-step approach to the process. We'll gradually expand model support and eventually transition to allowing users to define their own models.