Add an unsupervised warm up for the models - #140
Conversation
gabrieltseng
changed the title
|
This is super cool Gabi! Thanks so much. Just reviewing now |
tommylees112
left a comment
tommylees112
left a comment
There was a problem hiding this comment.
This is amazing work dude!! Just a few qs - thanks so much for implementing
| neighbour_indices: List[int] = [] | ||
| distant_indices: List[int] = [] | ||
|
|
||
| outer_distance = tuple(multiplier * val for val in distance) |
There was a problem hiding this comment.
whats the role of the multiplier?
There was a problem hiding this comment.
It's basically to enforce a minimum distance between the neighbouring instance and the distant instance.
The neighbour will be within neighbouring_distance of the anchor. The distant instance will be further than multiplier * neighbouring_distance from the anchor
There was a problem hiding this comment.
gotcha! so basically enforcing how large an area our spatial differences should be over
|
|
||
| x = self.rnn_dropout(hidden_state[:, -1, :]) | ||
|
|
||
| if return_embedding: |
There was a problem hiding this comment.
is this for interpreting the static embedding?
There was a problem hiding this comment.
No - the loss in tile2vec compares the embedding, not the final value. This is to return that embedding for the loss, before it gets put through the final linear layer
There was a problem hiding this comment.
yes makes sense! Could this be used for interpreting the embedding layer too though?
There was a problem hiding this comment.
yea, 100%. Although here the "embedding" is the final output of the model before the linear regression layer
| # initialize the model | ||
| if self.model is None: | ||
| x_ref, _, _ = next(iter(train_dataloader)) | ||
| model = self._initialize_model(self._input_to_tuple(x_ref)) |
There was a problem hiding this comment.
does this train the LSTM model? Don't we need to initialise with a CNN as they use in Tile2Vec?
There was a problem hiding this comment.
The principles of tile2vec can be used with any model that takes a raw input and outputs an embedding.
So yea, in this case it can also train the (EA)LSTM model
There was a problem hiding this comment.
okay gotcha.
So have i interpreted this correctly:
"We use the unsupervised learning algorithm described in Tile2Vec to pretrain (initialise) the weights of the EALSTM. This allows us to produce weights in the network that produce sensible spatial patterns. Mainly that pixels close together are more similar than pixels that are far apart."
There was a problem hiding this comment.
yea, that's exactly right
2 participants
Inspired by tile2vec, pretrains the models by training the models to make embeddings that are far away from each other more different than embeddings that are close to one another.
It's a less rigid way of communicating the latlon information to the models