Yixin Song
commited on
Commit
·
77ad0be
1
Parent(s):
9405b71
Update README.md
Browse files
README.md
CHANGED
|
@@ -59,7 +59,7 @@ We evaluate the model on the datasets of [Open LLM Leaderboard](https://huggingf
|
|
| 59 |
|
| 60 |
### Inference Tool
|
| 61 |
|
| 62 |
-
We utilize [PowerInfer](https://github.com/SJTU-IPADS/PowerInfer) for inference, here we present the inference speeds of pure CPU-based inference with fp16 precision. The CPU configuration includes an Intel i9-13900K processor (eight
|
| 63 |
|
| 64 |
Dense Inference: 0.52 tokens/s
|
| 65 |
|
|
|
|
| 59 |
|
| 60 |
### Inference Tool
|
| 61 |
|
| 62 |
+
We utilize [PowerInfer](https://github.com/SJTU-IPADS/PowerInfer) for inference, here we present the inference speeds of pure CPU-based inference with fp16 precision. The CPU configuration includes an Intel i9-13900K processor (eight performance cores at 5.4GHz) and 192GB of host memory (with a memory bandwidth of 67.2 GB/s).
|
| 63 |
|
| 64 |
Dense Inference: 0.52 tokens/s
|
| 65 |
|