The use of the checkpoint file in inference

[drdr-jiang]

Hello,

First of all, I would like to sincerely thank you for your work and for providing the code. I have a question to ask: if I create a custom dataset and train it, is the resulting ‘’checkpoint‘’ file only usable for evaluation? I used inference and noticed that the model options are set to "pinhole," "simple_radial," etc., but I didn't see my custom dataset-trained ''checkpoint'' file being used in the inference. Also, I didn't see how to specify the custom-trained checkpoint file in the inference process.

[veichta]

Hi,

Currently, we only support the original weights in the extractor, but I agree it would be useful to allow initialization from a custom checkpoint. I’ll work on implementing this feature as soon as possible.

In the meantime, you can try the following workaround:

import torch
from geocalib import GeoCalib
model = GeoCalib()

# Load the model from the checkpoint
checkpoint_path = "path/to/checkpoint" # should be something like 'outputs/training/<run name>'

state_dict = torch.load(f"{checkpoint_path}/checkpoint_best.tar", map_location="cpu")
model.model.flexible_load(state_dict["model"])

Note: This method will work only if there are no architectural changes in your configuration.

Let me know if you run into any issues!

[veichta]

You can now load your custom weights by passing the path to the corresponding checkpoint.

If you’ve made changes to the architecture, make sure to use the extractor from siclib:

from siclib.models.extractor import GeoCalib

checkpoint_path = "path/to/checkpoint" # usually something like 'outputs/training/<run name>/checkpoint_best.tar'
model = GeoCalib(weights=checkpoint_path)

[drdr-jiang]

Thank you very much for your answers and help!!! I will use your method and keep in touch with you in the future！

[drdr-jiang]

Hi,
After modifying the extractor.py file, how is "name" being called? I have extracted the checkpoint_best.tar file, and there is no config file inside. When loading the model, it cannot find conf["name"].

[veichta]

Hi, Could you maybe send the error message and the keys from the state_dict with print(state_dict.keys())?
Also make sure that you pass the path to the .tar file and not the extracted weights.

[drdr-jiang]

HI,
Currently, I have updated the extractor code you provided, and then I call the .tar file in my demo.py. The error is shown in the image below:

[veichta]

Hi, it seems that the config is present in the checkpoint. However, I made a mistake with how the config is accessed. This line should be added after loading the state_dict:
state_dict["conf"] = state_dict["conf"]["model"]

I've pushed a fix to the extractor where I also remove any weights used to initialize the training of the model.

[drdr-jiang]

Thank you very much for your help! ！！I have successfully run it, and I will continue with the next steps. I hope to stay in touch with you.