Spaces:
Running
on
Zero
Running
on
Zero
Update app.py
Browse files
app.py
CHANGED
|
@@ -260,26 +260,4 @@ with gr.Blocks(title="📖 Image/PDF OCR with LightOnOCR", theme=gr.themes.Soft(
|
|
| 260 |
|
| 261 |
|
| 262 |
if __name__ == "__main__":
|
| 263 |
-
demo.launch()
|
| 264 |
-
```
|
| 265 |
-
|
| 266 |
-
**Key improvements:**
|
| 267 |
-
|
| 268 |
-
1. **Conditional flash-attn installation**: Only installs flash-attn when CUDA is available
|
| 269 |
-
2. **Automatic attention selection**:
|
| 270 |
-
- **GPU**: `flash_attention_2` (fastest and most memory-efficient)
|
| 271 |
-
- **CPU**: `eager` (standard PyTorch attention, best for CPU)
|
| 272 |
-
3. **Appropriate dtype**: `bfloat16` for GPU, `float32` for CPU
|
| 273 |
-
4. **Performance optimizations**:
|
| 274 |
-
- Added `torch.no_grad()` context for inference
|
| 275 |
-
- Proper temperature handling (0.0 for greedy decoding)
|
| 276 |
-
5. **UI feedback**: Shows device and attention implementation in the interface
|
| 277 |
-
|
| 278 |
-
**Requirements.txt:**
|
| 279 |
-
```
|
| 280 |
-
gradio
|
| 281 |
-
torch
|
| 282 |
-
transformers>=4.37.0
|
| 283 |
-
pypdfium2
|
| 284 |
-
pillow
|
| 285 |
-
spaces
|
|
|
|
| 260 |
|
| 261 |
|
| 262 |
if __name__ == "__main__":
|
| 263 |
+
demo.launch()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|