Perfusion | Nvidia

Discover the innovative approach of Key-Locked Rank One Editing with Perfusion, a groundbreaking text-to-image personalization method. Introduced by researchers from NVIDIA and Tel Aviv University and accepted to SIGGRAPH 2023, this technology tackles the complex challenges of personalizing text-to-image models. With a small additional model size of just 100KB per concept and a brief 4-minute training period, Perfusion excels in producing creatively personalized objects, allowing significant visual alterations without losing the object's core identity. The Key-Locking mechanism is instrumental in maintaining a consistent identity across images, while also enabling the combination of several learned concepts into one image. Furthermore, Perfusion delivers flexibility at inference time, balancing visual and textual harmony with a single trained model, stretching across the entire Pareto front without extra training. The method impresses with both qualitative and quantitative improvements over existing models, offering a new way to portray personalized object interactions.

Key Features

• Efficient Model Size: A mere 100KB model size per concept for personalized text-to-image creation.

• Quick Training: Ability to train the model in approximately 4 minutes.

• Key-Locking Mechanism: Innovative feature that maintains identity during appearance changes.

• Combines Multiple Concepts: Capability to amalgamate individually learned concepts into a singular image.

• Visual and Textual Balance: Offers control over the trade-off between visual fidelity and textual alignment using a single model.

FAQ

What is Perfusion in text-to-image personalization?

Perfusion is a new method for text-to-image personalization that enables the portrayal of personalized objects with significant changes in appearance, while preserving their identity via a novel mechanism known as Key-Locking.

How does Perfusion avoid overfitting in personalized concepts?

The Perfusion architecture involves dynamic rank-1 updates to the underlying text-to-image model and introduces a Key-Locking mechanism to avoid overfitting personalized concepts to their superordinate category.

Which conference has Perfusion been accepted to?

Perfusion was accepted to SIGGRAPH 2023, notable for its contributions to graphics, interaction, and gaming technologies.

How large is the Perfusion model for each personalized concept?

Although the pre-trained model is several GBs, the additional size for each personalized concept in Perfusion is just 100KB.

What is the Key-Locking mechanism in Perfusion?

Key-Locking is a mechanism in the Perfusion model which helps in preserving the identity of personalized objects even when their appearance changes significantly.