IEEE

Self-Attention-Based Text Encoder for Enhancing DMGAN Text-to-Image Generation

Abstract: Generating images that align with textual input using text-to-image (TTI) generation models is a challenging task. Generative adversarial network (GAN) based TTI models can produce realistic ...

New Apple model combines vision understanding and image generation with impressive results

Manzano combines visual understanding and text-to-image generation, while significantly reducing performance or quality trade-offs.
GitHub

GD-Retriever: Controllable Generative Text-Music Retrieval with Diffusion Models

1 Centre for Digital Music, Queen Mary University of London, U.K. 2 Music & Audio Machine Learning Lab, Universal Music Group, London, U.K. Multimodal contrastive models have achieved strong ...
GitHub

ComfyUI supports over rhymes-ai/Allegro, which uses text prompt to generate short video in relatively high quality, especially comparing to other open source solutions ...

git clone https://github.com/bombax-xiaoice/ComfyUI-Allegro custom_nodes/ComfyUI-Allegro pip install -r custom_nodes/ComfyUI-Allegro/requirements.txt You can download ...
IEEE

Scaling Down Text Encoders of Text-to-Image Diffusion Models

Abstract: Text encoders in diffusion models have rapidly evolved, transitioning from CLIP to T5-XXL. Although this evolution has significantly enhanced the models’ ability to understand complex ...
marktechpost

Meta AI Open-Sourced Perception Encoder Audiovisual (PE-AV): The Audiovisual Encoder Powering SAM Audio And Large Scale Multimodal Retrieval

Perception Encoder, PE, is the core vision stack in Meta’s Perception Models project. It is a family of encoders for images, video, and audio that reaches state of the art on many vision and audio ...