Bapt120 commited on
Commit
9235b22
·
verified ·
1 Parent(s): e8397c3

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +1 -23
app.py CHANGED
@@ -260,26 +260,4 @@ with gr.Blocks(title="📖 Image/PDF OCR with LightOnOCR", theme=gr.themes.Soft(
260
 
261
 
262
  if __name__ == "__main__":
263
- demo.launch()
264
- ```
265
-
266
- **Key improvements:**
267
-
268
- 1. **Conditional flash-attn installation**: Only installs flash-attn when CUDA is available
269
- 2. **Automatic attention selection**:
270
- - **GPU**: `flash_attention_2` (fastest and most memory-efficient)
271
- - **CPU**: `eager` (standard PyTorch attention, best for CPU)
272
- 3. **Appropriate dtype**: `bfloat16` for GPU, `float32` for CPU
273
- 4. **Performance optimizations**:
274
- - Added `torch.no_grad()` context for inference
275
- - Proper temperature handling (0.0 for greedy decoding)
276
- 5. **UI feedback**: Shows device and attention implementation in the interface
277
-
278
- **Requirements.txt:**
279
- ```
280
- gradio
281
- torch
282
- transformers>=4.37.0
283
- pypdfium2
284
- pillow
285
- spaces
 
260
 
261
 
262
  if __name__ == "__main__":
263
+ demo.launch()