Automated Model Fine-Tuning and Deployment Using AWS SageMaker: A Scalable Workflow for Image Generation

Daniel Okwu

doi:10.63282/3050-9246.IJETCSIT-V3I1P101

Authors

Prof. Daniel Okwu West African Institute of Technology, Nigeria. Author

DOI:

https://doi.org/10.63282/3050-9246.IJETCSIT-V3I1P101

Keywords:

Automated model fine-tuning, AWS SageMaker, image generation, hyperparameter optimization, Conditional GAN, Inception Score, Fréchet Inception Distance, scalability, federated learning, multi-task learning

Abstract

The rapid advancement in deep learning and machine learning (ML) has led to significant improvements in various domains, including image generation. However, the process of fine-tuning and deploying these models remains challenging due to the complexity and resource requirements. This paper presents a scalable workflow for automated model fine-tuning and deployment using AWS SageMaker, focusing on image generation tasks. We describe the architecture, methodologies, and tools used to streamline the process, ensuring efficiency and scalability. The paper also includes experimental results and a comparative analysis with traditional methods, demonstrating the effectiveness and efficiency of our proposed approach.

Downloads

Download data is not yet available.

References

[1] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems (pp. 2672-2680).

[2] Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114.

[3] Amazon Web Services. (2021). AWS SageMaker. Retrieved from https://aws.amazon.com/sagemaker/

[4] Zhang, H., Goodfellow, I., Metz, L., & Odena, A. (2018). Self-attention generative adversarial networks. arXiv preprint arXiv:1805.08318.

[5] Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in neural information processing systems (pp. 6626-6637).

[6] https://docs.aws.amazon.com/sagemaker-unified-studio/latest/userguide/bedrock-image-playground-generate-image.html

[7] https://www.pluralsight.com/resources/blog/ai-and-data/deploying-genAI-amazon-sagemaker

[8] https://docs.aws.amazon.com/sagemaker/latest/dg/studio-byoi-create.html

[9] https://aws.amazon.com/blogs/machine-learning/automate-fine-tuning-of-llama-3-x-models-with-the-new-visual-designerfor-amazon-sagemaker-pipelines/

[10] https://www.youtube.com/watch?v=2t4Qiq7hhJ8

[11] https://docs.aws.amazon.com/sagemaker/latest/dg/canvas-fm-chat-fine-tune.html

[12] https://aws.amazon.com/blogs/machine-learning/prepare-image-data-with-amazon-sagemaker-data-wrangler/

[13] https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-fine-tune.html

[14] https://aws.amazon.com/blogs/machine-learning/generate-unique-images-by-fine-tuning-stable-diffusion-xl-with-amazonsagemaker/

[15] https://aws.amazon.com/blogs/machine-learning/architect-personalized-generative-ai-saas-applications-on-amazonsagemaker/