[WIP] Gromov-Wasserstein with asymmetric cost matrices

[decarpentierg]

Types of changes

• Support of a wider class of arguments for Gromov-Wasserstein functions
• Signature of 2 functions modified
• Docstrings updated

Motivation and context / Related issue

In [12] "Gromov-Wasserstein Averaging of Kernel and Distance Matrices", the authors make no explicit assumption about the symmetry of the cost mastrices C and C'. However, their formula (9) for the computation of the gradient misses at least a 2 factor in the case of symmetric matrices, and even a second term in the case of asymmetric matrices, as explained in subsection 5.1 of "Multilingual Alignment of Word Embedding Spaces", Dan Meller and Gonzague de Carpentier, 2021 (https://www.researchgate.net/publication/357505092_Multilingual_alignment_of_word_embedding_spaces). In the code of ot/gromov.py, the 2 factor has been corrected, but it is not sufficient to support asymmetric cost matrices, which can be useful in the case of directed graphs for example. This contribution corrects the expression of the gradient in ot.gromov.gwggrad to support asymmetric distance matrices, by adding the appropriate term instead of just multiplying by 2.

How has this been tested (if it applies)

PR checklist

• I have read the CONTRIBUTING document.
• The documentation is up-to-date with the changes I made (check build artifacts).
• All tests passed, and additional code has been covered with new tests.
• I have added the PR and Issue fix to the RELEASES.md file.

[codecov]

Codecov Report

Merging #399 (1eb9aab) into master (b295ffc) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #399      +/-   ##
==========================================
- Coverage   93.97%   93.94%   -0.04%     
==========================================
  Files          22       22              
  Lines        5927     5930       +3     
==========================================
+ Hits         5570     5571       +1     
- Misses        357      359       +2     

[cedricvincentcuaz]

Hello decarpentierg.
Thank you for pointing out and fixing this limitation of the current gromov-wasserstein solver with your PR.
I plan to do a major update of the optim.py file to introduce a generic conditional gradient solver merging the current cg (conditional gradient) function and gcg (generalized conditional gradient). So any kind of linear program solver for the direction finding steps and line-search solver can be integrated more easily in the future. For this matter I also have to modify the gromov functions of gromov.py that you are working on and will integrate new functions for instance for the semi-relaxed gw divergence.
To avoid any conflict, do you mind if I start building on your code and create a new/bigger pull-request ? I can also handle the tests currently missing for the merge of your PR, as I will have to introduce new parameters for the MAJ I envision.

Best regards,
Cédric

[decarpentierg]

Hello Cédric,
Yes feel free to to do whatever you want with this PR, I don't intend to work further on it.
Best regards,
Gonzague

[rflamary]

Thanks again @decarpentierg for the PR. As discussed above the PR #431 of @cedricvincentcuaz is advancing and contains a working imlementtaion with the non symmetric marices. I'm closing this PR now.