ShipSense

Novel few-shot synthetic image data augmentation method using fine-tuned Stable Diffusion for any object. ShipSense used custom Stable Diffusion to create realistic synthetic image data of ships and trained a convolutional neural networks (CNNs) to detect and locate ships from satellite imagery. They also built a data visualization platform for stakeholders to monitor overfishing. To enhance this platform, they identified several hotspots of suspicious dark vessel activity by digging into 55,000+ AIS radar records.

While people have tried to build AI models to detect overfishing before, accuracy was poor due to high class imbalance. There are few positive examples of ships on water compared to the infinite negative examples of patches of water without ships. Researchers have used GANs to generate synthetic data for other purposes. However, it takes around 50,000 sample images to train a decent GAN. The largest satellite ship dataset only has ~2,000 samples.

They realized that Stable Diffusion (SD), a popular text-to-image AI model, could be repurposed to generate unlimited synthetic image data of ships based on relatively few inputs. They were able to achieve highly realistic synthetic images using only 68 original images.