Dimension of the features

[h1manshus1ngh]

Can you please tell me which dimension features it is extracting?

[h1manshus1ngh]

If i want to extract 1024 dimensional features what i have to do?

[h1manshus1ngh]

Also i want to store features as .npy format in numpy array how can i do it?

[AlexMetsai]

Hi there, thanks for your interest in my code!

I can't know for sure from the feature's dimension alone, but I'm guessing that you 're probably looking for features extracted using the GoogLeNet neural network architecture. However, this repository concerns the ResNet architectures. I could extend it to include googlenet, but unfortunately I don't know when I'll have the time to do so. In any case, I think that this won't be hard to do, you should definitely give it a try! Anyway, for this code, the output of resnet's pool5 layer (the features you 're looking for) is of size 2048.

To save as .npy format, you can convert the tensors to numpy arrays and then simply save them in this fortmat, like seen in the pseudocode below.

temp = some_tensor.numpy()
np.save("filename.npy", temp)

I hope these will be easy to implement!

Best,
Alex

[h1manshus1ngh]

How many frames its taking for a single video to extract the features

[AlexMetsai]

That depends on the video's total number of frames (in manner, it depends on its duration). Expect a longer video to have a larger number of frames, but keep in mind that this actually depends on the video's fps number (frames per second).

[h1manshus1ngh]

is it extracting 3d features or 2d features from video i want to extract 3d features from UCF101 with the help of resnext101 but as i am exchanging the code is shows model not found.

[h1manshus1ngh]

i can manage with 2048 dimension features.

[AlexMetsai]

If you consider that for each frame you get a feature vector, then you probably want to represent each video as a 2D representation with dimensions (num_of_frames, 2048). Else, maybe you want to use an architecture that utilizes 3D convolutions? This is not the case with these models, where each frame is treated separately.

Are for the 'model not found' error, try with:

resnet = models.resnext101_32x8d(pretrained=True)

I think that this is the only available resnext101 model that the torchvision package provides.

[h1manshus1ngh]

Yes i want to use architecture that utilises 3D convolutions like resnext101.
how can i do. i have seen some git hub code available but they are not extracting features from each frame. How can i do that.

[AlexMetsai]

I don't know if resnext101 uses 3D convolutions in its vanilla version (reference).

As far as I know, 3D convolutions can be useful in utilizing information by looking at previous and next frames, therefore the features for each frame will contain information from these frames too. Maybe in the future I could build something like this for fun, but don't find it highly likely since 1) it would take some time and 2) I would probably explore more SotA approaches that utilize Transformers instead of 3D Convolutions, as a new project.

Video Transformer Network
Video Action Transformer Network

Anyway, good luck. :-)