Continual Diffusion

C-LoRA enhances text-to-image diffusion models for continual customization, enabling sequential learning of fine-grained concepts without forgetting previous ones. The method introduces a continually self-regularized low-rank adaptation, requiring no storage of user data for replay.

Key Features

• Continual Learning: C-LoRA allows continual customization of text-to-image diffusion models by sequentially learning new concepts without forgetting past ones.
• Low-Rank Adaptation: The method uses a continual, self-regulating low-rank weight adaptation in cross-attention modules to update model parameters.
• Random Feature Initialization: C-LoRA initializes custom tokens as random features and removes the concept name from the prompt, enhancing model flexibility.

Use Cases

• Mobile App: A mobile app sequentially learns new customized concepts, allowing users to generate photos of previously learned concepts at a later time.
• Fine-Grained Customization: C-LoRA enables fine-grained customization of text-to-image models, making them suitable for applications where specific details matter.
• Concept Learning and Generation: The method can be used to teach models new concepts and generate images based on those concepts.

Conclusion

The C-LoRA method enables continual customization of text-to-image diffusion models with sequential learning of new concepts, overcoming the catastrophic forgetting issue. C-LoRA demonstrates state-of-the-art results, positioning it as a promising solution for a wide range of applications, including continual learning in image classification and continual customization in text-to-image generation.