ggerganov commited on
Commit
6d07dee
·
1 Parent(s): 6d7599e

metal : enable mat-vec kernels for bs <= 4 (llama/10491)

Browse files
Files changed (1) hide show
  1. ggml/src/ggml-metal/ggml-metal.m +1 -1
ggml/src/ggml-metal/ggml-metal.m CHANGED
@@ -1935,7 +1935,7 @@ static void ggml_metal_encode_node(
1935
 
1936
  // find the break-even point where the matrix-matrix kernel becomes more efficient compared
1937
  // to the matrix-vector kernel
1938
- int ne11_mm_min = 1;
1939
 
1940
  #if 0
1941
  // the numbers below are measured on M2 Ultra for 7B and 13B models
 
1935
 
1936
  // find the break-even point where the matrix-matrix kernel becomes more efficient compared
1937
  // to the matrix-vector kernel
1938
+ int ne11_mm_min = 4;
1939
 
1940
  #if 0
1941
  // the numbers below are measured on M2 Ultra for 7B and 13B models