claw-eval/claw-eval#14
Maybe it's already here, but if not, it'd be a killer app. Basically, before tasking your agent have it get a the score for the task and you can set it to some level of quality (perhaps % of max score). It'd cache of course but new tasks are being added all the time...
The point is, if the task can be done for 0.001 at 95% of the quality of the best model (which is very expensive), you can end up saving an extreme amount of $$$
You can imagine, of course, where this might lead. Cheap models tuned to various tasks, and suddenly it's all near free. I'm sure you have this in mind..
This is a massive, massive opportunity. If you add cost, you could enable a meta 'MoE' and allow agentic flows to route along quality/cost in a way that competes with frontier models.
This is why Jensen is so excited about openclaw. Nvidia has been researching exactly this for a very long time.
claw-eval/claw-eval#14
Maybe it's already here, but if not, it'd be a killer app. Basically, before tasking your agent have it get a the score for the task and you can set it to some level of quality (perhaps % of max score). It'd cache of course but new tasks are being added all the time...
The point is, if the task can be done for 0.001 at 95% of the quality of the best model (which is very expensive), you can end up saving an extreme amount of $$$
You can imagine, of course, where this might lead. Cheap models tuned to various tasks, and suddenly it's all near free. I'm sure you have this in mind..
This is a massive, massive opportunity. If you add cost, you could enable a meta 'MoE' and allow agentic flows to route along quality/cost in a way that competes with frontier models.
This is why Jensen is so excited about openclaw. Nvidia has been researching exactly this for a very long time.