convert: fix get block count error for Nemotron 3 Ultra

[rockchenn]

Overview

It shows the below error while converting nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4.

$ python3 convert_hf_to_gguf.py --verbose --dry-run NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4
INFO:hf-to-gguf:Loading model: NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4
INFO:hf-to-gguf:Model architecture: NemotronHForCausalLM
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: indexing model part 'model-00001-of-00113.safetensors'
...
INFO:hf-to-gguf:heuristics detected bfloat16 tensor dtype, setting --outtype bf16
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
Traceback (most recent call last):
  File "/home/rock_chen/Downloads/llama.cpp/convert_hf_to_gguf.py", line 307, in
    main()
  File "llama.cpp/convert_hf_to_gguf.py", line 281, in main
    model_instance = model_class(dir_model, output_type, fname_out,
  File "llama.cpp/conversion/nemotron.py", line 216, in init
    super().init(*args, **kwargs)
  File "llama.cpp/conversion/granite.py", line 299, in init
    super().init(*args, **kwargs)
  File "llama.cpp/conversion/mamba.py", line 115, in init
    super().init(dir_model, *args, hparams=hparams, **kwargs)
  File "llama.cpp/conversion/llama.py", line 36, in init
    super().init(*args, **kwargs)
  File "llama.cpp/conversion/base.py", line 1145, in init
    self.block_count = self.find_hparam(["n_layers", "num_hidden_layers", "n_layer", "num_layers"])
  File "llama.cpp/conversion/granite.py", line 361, in find_hparam
    return Mamba2Model.find_hparam(self, keys, *args, **kwargs)
  File "llama.cpp/conversion/base.py", line 204, in find_hparam
    raise KeyError(f"could not find any of: {keys}")
KeyError: "could not find any of: ['n_layers', 'num_hidden_layers', 'n_layer', 'num_layers', 'mamba_n_layers', 'mamba_num_hidden_layers', 'mamba_n_layer', 'mamba_num_layers']"

In configuration_nemotron_h.py, it explains

    num_hidden_layers (`int`, *optional*):
       Number of hidden layers in the Transformer encoder. This parameter is deprecated and only kept for
       backward compatibility. The number of layers is now determined by the length of `layers_block_type`.

So I followed this instruction in this commit for NemotronH. Note that when layers_block_type isn't available, the script would also try legacy way for backward compatibility.

Requirements

• I have read and agree with the contributing guidelines

• AI usage disclosure: YES. AI was used for data search and code review.

[CISC]

This is specifically for this model only, please don't touch other code.

You can fix this in NemotronHModel.__init__ instead.

[rockchenn]

@CISC
Thank you for the feedback. The commit is updated.

Update:
Mamba2 has this requirement, but the current method may violate it. I'll check this later.
# Avoid using AutoConfig for hparams
# It wrongly assumes all Mamba2 models are Mamba-Codestral-7B-v0.1

Update:
The above comment was left in llama : initial Mamba-2 support- #9126. It seems some Mamba2 models have at least file renaming issue and missing mandatory key/value in config.json at that time.

Although Nemotron is a hybrid Mamba-Transformer MoE model, pure Mamba2 model conversions would not run any code in nemotron.py and would not be impacted by this change.