josharian commited on
Commit
1316242
·
unverified ·
1 Parent(s): 6ea3354

whisper : make beam candidate sort more stable (#1943)

Browse files

All else being otherwise equal, this encourages the beam candidate
selection to re-use the same decoder, which slightly
reduces the cache size.

I wouldn't expect it to make much of a performance difference,
but it helps when debug printing the cache and beam.

Added as part of understanding #1941.

Files changed (1) hide show
  1. whisper.cpp +4 -1
whisper.cpp CHANGED
@@ -5357,7 +5357,10 @@ int whisper_full_with_state(
5357
  beam_candidates.begin(),
5358
  beam_candidates.end(),
5359
  [](const beam_candidate & a, const beam_candidate & b) {
5360
- return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
 
 
 
5361
  });
5362
 
5363
  uint32_t cur_c = 0;
 
5357
  beam_candidates.begin(),
5358
  beam_candidates.end(),
5359
  [](const beam_candidate & a, const beam_candidate & b) {
5360
+ if (a.sequence.sum_logprobs_all != b.sequence.sum_logprobs_all) {
5361
+ return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
5362
+ }
5363
+ return a.decoder_idx < b.decoder_idx;
5364
  });
5365
 
5366
  uint32_t cur_c = 0;