Mecanismo de IA Generativo Controlável para Expandir a Automação na Deteção de Pragas Agrícolas

Data-related bottlenecks are blocking the widespread of automatic pest detection tools for unaddressed crops. These issues are prevalent across many application areas, contributing to the recent surge in CV-GenAI that sparked a wave of innovative data-centric initiatives based on synthetic image generation [13], but few transitioned to tangible impact on specialized domains [14]. In the AgriPestForge, we aim to contribute to the CV-GenAI field by improving how general-purpose SD models can be effectively customized and controlled for domain-specific tasks and be subsequently used to boost the usage of synthetic data in the development of new automatic pest detection tools for different crops (see Fig.2). We will focus on: 1. Improve Few-shot Adaptation of SD Models for Agricultural Pests: Contribute with new methods for domain-specific adaptation of pre-trained general-purpose SD models in few-shot settings (i.e., limited labeled data available). We will research more efficient and effective fine-tuning strategies to distill in-domain knowledge on the different sub-components of SD model's architecture - the VAE, the tex-encoder, and the U-Net - to improve its representation capabilities of target pests. 2. Improve Control over SD Generative Process for Agricultural Pests: Explore the adaptation of new controllable generation strategies [16, 17] to leverage the repurpose of SD models for domain-specific tasks. We will study a wide range of text- and image-based conditioning mechanisms to achieve fine-grained control of the image generation process at inference time, namely: i) trap-oriented guidance - trap types (e.g., delta, chromotropic or McPhail), image acquisition modality (static setup or handheld acquisition), illumination and weather conditions (e.g., shadows, reflections, rain), artifacts (e.g., dirt, leaves); and ii) insect-oriented guidance - key insect species, insect morphological variations (e.g., rotations, decomposition, wing positions), variable pest densities (from zero to hundreds of target insects in a single trap). 3. Create a novel Pest-tailored Controllable Generative AI Engine: Develop a tool that generates high-quality and diverse synthetic image datasets of target insect species in sticky traps. It will be built upon the envisioned contributions described above to streamline the generation of representative synthetic image datasets that accurately mimic the target trap-related and insect-related characteristics. These synthetic datasets will be used to augment and stress-test the supervised ML pest detection pipelines to be addressed in the validation use cases. 4. Validation Use Cases: We intend to harness and showcase the potential of the generated synthetic data to develop and audit automatic pest detection for different crops. In particular, we will address three scenarios in cooperation with key stakeholders: i) a Stress Testing Scenario for Viticulture; ii) a Low-data Scenario for Tomato Cultivation (Data Augmentation); and iii) a Domain Transfer Scenario for Olive grove. To assess the quality and impact of the synthetic data on these use cases, we will perform multidisciplinary research to evaluate them qualitatively (e.g., feedback from taxonomy specialists and stakeholders) and quantitatively (e.g., fidelity, diversity). From a utility perspective, we aim to do comparative studies on the performance of different pest detection predictive models trained with varying levels of synthetic data.