This repository hosts the code and data for the research conducted in An Empirical Study of Challenges in Machine Learning Asset Management by Zhimin Zhao et al. The study explores the multifaceted challenges of managing machine learning (ML) assets, drawing data from diverse online discussion forums.

Watch the following video teaser to learn about our motivation, methodology, and key findings!

Image: GPT-4o

Music: Suno

Video: Canva

https://github.com/user-attachments/assets/8564059e-f5a6-41bf-bb85-c060ee59ddf7

Overview

Our study comprehensively analyzes data from general forums (e.g., Stack Overflow), repository-specific forums (e.g., GitHub), and tool-specific forums (e.g., DVC) to uncover prevalent challenges and emerging solutions in ML asset management.

Contents

1. Code: The repository includes scripts for data collection, preprocessing, and analysis, primarily developed for topic modeling. Notably, the data mining scripts from repository-specific forums can be accessed at MSR Asset Management, and the general and tool-specific forum counterparts at QA Asset Management.
2. Data: This repository includes the raw dataset sourced from various forums, alongside the curated and preprocessed datasets.
3. Results: This repository includes the results and graphs from various analyses of the curated datasets.

Citation

Please cite our work as follows if you utilize this repository:

@article{zhao2024empirical,
  title={An Empirical Study of Challenges in Machine Learning Asset Management},
  author={Zhao, Zhimin and Chen, Yihao and Bangash, Abdul Ali and Adams, Bram and Hassan, Ahmed E},
  journal={Empirical Software Engineering},
  year={2024}
}

Contact

For queries or collaboration, feel free to raise an issue.