Commit ef2bd45
authored
Removing L2-norm in contrastive loss (L2-norm already present in CosSim) (#6550)
### Description
The `forward` method of the `ContrastiveLoss` performs L2-normalization
before computing cosine similarity. The
[`torch.nn.functional.cosine_similarity`](https://pytorch.org/docs/stable/generated/torch.nn.functional.cosine_similarity.html)
method already handles this pre-processing to make sure that `input` and
`target` lie on the surface of the unit hypersphere. This step involves
an unnecessary cost and, thus, can be removed.
### Types of changes
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [x] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [x] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
Signed-off-by: Lucas Robinet <robinet.lucas@iuct-oncopole.fr>1 parent 8dd004a commit ef2bd45
1 file changed
Lines changed: 1 addition & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | 71 | | |
75 | 72 | | |
76 | 73 | | |
77 | | - | |
| 74 | + | |
78 | 75 | | |
79 | 76 | | |
80 | 77 | | |
| |||
0 commit comments