Ultra.
Streamlabs, Inworld AI, and NVIDIA are excited to unveil our new intelligent streaming assistant. This AI Agent leverages cutting-edge technology developed by Logitech G’s Streamlabs, in collaboration with NVIDIA, and Inworld to simplify the live streaming experience by giving the streamer their very own production team. This includes
- 3D sidekick: Interactive co-host that can answer questions about the game or the stream, research facts, and help get the conversation going when the stream is a little quiet by commenting on in-game events and chat activity. Ask the sidekick anything from game information to chat activity, and your sidekick will provide intelligent, context-aware answers.
- Producer: Automatically adjusts scenes based on what’s happening in stream, chat, or by the streamer’s request. Plays video and audio cues during gameplay to add tension or excitement to the stream. Clips your best moments and plays replays on stream at the streamer's request.
- Technical assistant: Get help setting up Streamlabs Desktop, setting up exciting scene transitions, or troubleshooting technical issues. Your AI-powered assistant can also monitor your stream to ensure it runs smoothly.
We formally unveiled this intelligent streaming assistant at CES 2025, with an anticipated release projected within the first half of 2025. Streamlabs invites the community to stay informed on updates by visiting the Streamlabs website and blog or connecting on X and Discord, as it continues to enhance and refine this transformative tool.
Why did we build this?
We know streaming can be overwhelming for new streamers, as they must manage technical setups, engage with viewers, and create an entertaining experience. With a limited audience, it can be challenging to foster interaction, especially when chat activity is low, and viewers may feel hesitant to participate. Moreover, streamers are more than just entertainers; they are multitaskers, balancing gaming, live entertainment, and troubleshooting.
Our intelligent streaming assistant changes that by helping the streamer manage all aspects of your live stream. This AI agent can join you on stream as a 3D sidekick and interact with you and your chat, as well as adjust scenes, clip your best moments, and troubleshoot technical issues in the background.
Ask it to perform tasks, including starting and stopping your stream, changing scenes, muting audio sources, clipping your stream, playing your latest replay, posting to your chat, and much more. Furthermore, the assistant's ability to incorporate reactive overlays and filters based on gameplay performance enriches viewer immersion, so you don’t lose your focus on the game.
The sidekick can help you during slow chat times by offering comments on your gameplay when things are too quiet, giving you someone to chat with or ask questions, and also responding in chat to general questions from viewers like a chatbot would - but with interesting responses thanks to the powerful LLMs behind it.
Customizable personality options—ranging from supportive to sarcastic avatars— are also available allowing streamers to tailor the assistant's interactions to align with their unique style and audience engagement goals.
Collaborating For Innovation: Streamlabs, Logitech, NVIDIA, and Inworld Creating the Future of Interactive Streaming
Through this partnership, Streamlabs is committed to transforming the digital broadcasting landscape, working with technology partners like NVIDIA and Inworld to continuously innovate and improve the streaming experience for content creators.
Streamlabs is developing the user-facing app that will live within the Streamlabs App Store and serve as the interface for the virtual assistant. Avatar rendering and animation is powered by NVIDIA ACE technologies. NVIDIA ACE also powers the AI vision language models to analyze your gameplay and provide context to your sidekick or producer. Inworld functions as the cognitive engine, processing inputs and executing commands through the Streamlabs API.
Together, these three companies have combined their expertise and resources to create an intelligent streaming assistant that will simplify the live streaming experience for streamers. By working together, we aim to bring cutting-edge technology directly into the hands of content creators.
FAQ:
- When will this be available?
- We are unveiling this new tool at CES 2025. However, we have yet to set an exact date for general availability, but we estimate it will be released in the first half of 2025. Stay tuned for more updates!
- How do I use this intelligent streaming assistant?
- To use the intelligent streaming assistant, you will need to have a Streamlabs account and download the user-facing app from the Streamlabs App Store. Once downloaded, you can activate and customize your intelligent virtual assistant to interact with your live stream in real-time.
- What tasks can this virtual assistant perform?
- This AI stream assistant can automatically handle manual tasks such as clipping game highlights, changing scenes, and posting in chat while also reacting to gameplay moments and responding to viewer questions. You can also use voice commands to execute actions like starting or stopping your stream, muting audio sources, and playing replays.
- How does the avatar know what's happening in my game?
- The intelligent virtual assistant is powered by Inworld AI, which processes inputs from your game and executes commands through the Streamlabs API. This allows the assistant to dynamically understand the gameplay on your screen and react accordingly. NVIDIA AI Blueprints power rendering avatars seamlessly on screen.
- Who are the partners involved in creating this intelligent streaming assistant?
- The intelligent virtual assistant is a collaborative effort between Streamlabs, NVIDIA, and Inworld. Each partner brings their expertise in developing user-facing apps, rendering avatars seamlessly on screen, and providing the cognitive engine for real-time interactions. Together, we aim to revolutionize the live streaming experience for both streamers and viewers.
- Is this free?
- Our product will operate on a freemium model, offering users a limited amount of avatar time for free—approximately one to two hours. Beyond this complimentary period, users will need to subscribe to our Ultra plan to access additional features and extended usage.
- Can I customize my avatar's appearance?
- Yes, you can choose from a variety of avatars and customize their personality and appearance to fit your brand.
- Which games does this work with?
- At launch, it will work with Fortnite. Ultimately, we plan to expand support to other popular games.
- Can I have multiple avatars interacting in my stream?
- Currently, only one avatar can interact with your stream at a time. However, we are continuously working to improve and expand features for our users.
- Will this virtual assistant work on all streaming platforms?
- Yes, it will work on all platforms.
- What advantages does utilizing a GeForce RTX GPU
- Running NVIDIA ACE locally on your GeForce RTX GPU detects in game events with lower latency, letting the producer or sidekick react quicker. Rendering the sidekick locally makes interactions with the streamer more natural and responsive.