Autonomous Multimodal Content Generation Using Open-Source LLMs, Diffusion Models, and Cross-Lingual TTS

• 1Anna University Chennai, UG Student, Department of Artificial Intelligence and Data Science, Prathyusha Engineering College, Tiruvallur , IN
• 2Anna University Chennai, Student, Department of Artificial Intelligence and Data Science, Prathyusha Engineering College, Tiruvallur, IN
• 3Anna University Chennai, Assistant professor, Department of Artificial Intelligence and Data Science, Prathyusha Engineering College Tiruvallur, Chennai, IN

The rapid evolution of online content creation demands innovative, autonomous solutions capable of generating high-quality, multimodal content across diverse social media platforms. This research presents a groundbreaking approach to content generation that leverages advanced open-source large language models (LLMs) such as Llama 3.2 70B, Llama-3.1-Nemotron-70B-Instruct, and Gemini Flash 002, for automated and versatile text generation. Complemented by state-of-the-art web scraping and web search tools, our framework continuously mines real-time data to adapt content relevance and engagement dynamically. For controlled, high-fidelity image synthesis, our methodology integrates a Guided Diffusion Framework, allowing precise manipulation of visual elements to align with brand aesthetics or thematic requirements. A significant innovation of our approach is including Cross-Lingual Zero-Shot Text-to-Speech (TTS) synthesis, employing supervised semantic tokens for natural voice cloning and synthesis. This TTS model facilitates multilingual voice generation and achieves 20–35% higher fidelity and naturalness compared to existing baselines. This capability enables seamless voice synchronization for generated content, enhancing the accessibility and appeal of multimedia content across linguistic barriers. Our tool also interfaces with social media APIs to streamline content posting, monitor performance analytics, and recommend optimized content strategies. Combining these multimodal generative models and real-time web-sourced insights, our approach redefines content creation potential, setting a new standard in AI-driven, autonomous multimedia generation.