Creating images of imaginative worlds using generative AI can be a challenge to control using text-to-image alone. At #uist2023 Hai Dang will present #WorldSmith today, work we did while he was interning with us at Autodesk Research in Toronto (with @fraseranderson, George Fitzmaurice, and me).
WorldSmith is a novel UI & workflow for creators to composite scenes using iterative, multi-modal prompts for #generativeAI. It allows creators to specify their intent through text, sketching, or region-based input.
Images can be blended together to form one cohesive depiction of a fictional world.
The progress is tracked in an interactive graph, offering a dynamic way to explore and evolve their creations.
#WorldSmith conceptualizes two expressive prompting techniques for #generativeAI: hierarchical prompting (which attaches prompts to different layers of the composition) and spatial prompting (allowing users to specify spatial relations through direct input).
Hai will present #WorldSmith today at #uist2023 at 3:02pm Pacific Time.
Join live or watch the live stream https://programs.sigchi.org/uist/2023/program/content/126668
30s video preview: https://youtube.com/watch?v=U1iF6GVbHL4
Video figure: https://youtube.com/watch?v=MBO9Uen597w
Paper: https://research.autodesk.com/publications/worldsmith/