🔖 ImageBind: One Embedding Space To Bind Them All

📝 This project is a significant step forward in understanding and connecting information from diverse sources like images, text, audio, video, and even motion sensor data.

⚙️ Supports 6 Modalities:

📷 Image
📝 Text
🔈 Audi
🎥 Video
🦴 IMU sensor data (e.g., accelerometer)
🙄 Depth/Thermal & 3D data
Interestingly, only some modalities had labels, yet ImageBind learned to align them through self-supervised learning.


💫 Key Features:

..No need for paired data (e.g., images and audio don’t have to be aligned)..Leverages contrastive learning for learning joint embedding space
..Competes with CLIP and AudioCLIP, but with better accuracy and coverage..Enables zero-shot retrieval (e.g., finding relevant video using just a sentence)


📌 Repo: https://github.com/facebookresearch/ImageBind

🔍 By: https://yangx.top/DataScienceN 🌟

#ImageBind #MultimodalAI #MetaAI #DeepLearning #SelfSupervised

🐈‍⬛ TTT Long Video Generation 🐈‍⬛

▶️ A novel architecture for video generation, adapting the #CogVideoX 5B model by incorporating #TestTimeTraining (TTT) layers.
Adding TTT layers into a pre-trained Transformer enables generating a one-minute clip from text storyboards.
Videos, code & annotations released 💙

🔗 Review: https://t.ly/mhlTN
📄 Paper: arxiv.org/pdf/2504.05298
🌐 Project: test-time-training.github.io/video-dit
🧑‍💻 Repo: github.com/test-time-training/ttt-video-dit

#AI #VideoGeneration #MachineLearning #DeepLearning #Transformers #TTT #GenerativeAI

🔍 By: https://yangx.top/DataScienceN5