In QNetwork and DuelingQNetwork brains the optimizer is getting initialized at every update step, which makes the training significantly slower:
def update(self, _, pred, target):
optimizer = torch.optim.Adam(self.parameters(), lr=self.learning_rate)
Maybe a solution will be to convert it into a class attribute. Ej.:
class QNetwork(nn.Module, Brain):
def __init__(self, env: gym.Env, hidden_layers=[], learning_rate=0.01, alpha=0.001):
...
self.optimizer = torch.optim.Adam(self.parameters(), lr=self.learning_rate)
In
QNetworkandDuelingQNetworkbrains the optimizer is getting initialized at every update step, which makes the training significantly slower:Maybe a solution will be to convert it into a class attribute. Ej.: