bilalfaye commited on
Commit
b196338
Β·
verified Β·
1 Parent(s): 50f445f

OneEncoder for text and image with temperature = 2.5 during training.

Browse files
Files changed (2) hide show
  1. README.md +3 -31
  2. model.safetensors +2 -2
README.md CHANGED
@@ -2,36 +2,8 @@
2
  tags:
3
  - model_hub_mixin
4
  - pytorch_model_hub_mixin
5
- license: mit
6
- language:
7
- - en
8
- base_model:
9
- - openai/clip-vit-large-patch14
10
- - google-bert/bert-base-uncased
11
  ---
12
 
13
- # πŸ–ΌοΈπŸ“ OneEncoder: A Unified Text & Image Model
14
-
15
- **OneEncoder** is a lightweight framework for cross-modal alignment, focusing on efficiently integrating **text and images** (with future extensions to other modalities). Unlike traditional methods relying on massive modality-specific encoders, OneEncoder progressively aligns different data types, making it cost-effective and performant even on small paired datasets.
16
-
17
- ## πŸš€ Key Features
18
- βœ… **Multimodal Alignment**: Initially supports **text & image**, with extension to other modalities.
19
- βœ… **Lightweight & Efficient**: Avoids full retraining when adding new modalities.
20
- βœ… **Superior Performance**: Outperforms models that require large specialized datasets.
21
-
22
- ## 🎯 Applications
23
- - **Visual Question Answering (VQA)**
24
- - **Image-Text Retrieval**
25
- - **Multimodal Content Understanding**
26
-
27
- ## πŸ“ Authors
28
- πŸ“Œ **Bilal FAYE**, Hanane AZZAG, Mustapha LEBBAH, Djamel BOUCHAFFRA
29
-
30
- ## πŸ“„ Research Paper
31
- πŸ“œ **arXiv**: [OneEncoder: Progressive Cross-Modal Alignment](https://arxiv.org/abs/2409.11059)
32
-
33
- ## πŸ“Œ Resources
34
- πŸ”— **GitHub Repo**: [OneEncoder](https://github.com/b-faye/OneEncoder)
35
- πŸš€ **Hugging Face Demo**: [OneEncoder Retriever](https://huggingface.co/spaces/bilalfaye/OneEncoder-retriever)
36
- πŸ““ **Demo Notebook**: [OneEncoder Demos](https://github.com/b-faye/OneEncoder/tree/main/demo)
37
- πŸ”Š **OneEncoder for Text, Image & Audio**: [HF Model](https://huggingface.co/bilalfaye/OneEncoder-text-image-audio)
 
2
  tags:
3
  - model_hub_mixin
4
  - pytorch_model_hub_mixin
 
 
 
 
 
 
5
  ---
6
 
7
+ This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
8
+ - Library: [More Information Needed]
9
+ - Docs: [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2ddc7a649ca5ad6043061f4a29d11bb0fa76393b1b3ba296c6f94046ca65dd7f
3
- size 834761380
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:25c9070a5a75f48a5c6cd7c56635db8f25352192ee08adead0f5e68643ac2a7f
3
+ size 793972836