AI for Intelligent Data Placement in Distributed File Systems: Optimizing Throughput and Latency in Edge-Cloud Environments
Keywords:
Edge Computing, Artificial Intelligence, Reinforcement Learning, Distributed File SystemsAbstract
With the rapid rise in data generation and the proliferation of edge devices, optimizing data placement in distributed file systems has become a critical challenge, especially when considering the interplay between edge and cloud computing environments. Artificial intelligence (AI) offers transformative potential in enhancing data placement strategies by improving both throughput and latency. This paper explores the application of AI-driven methods for intelligent data placement in distributed file systems within edge-cloud environments. By analyzing key AI techniques such as reinforcement learning (RL), machine learning (ML), and deep learning (DL), this paper discusses how these methods can optimize data access speed, minimize latency, and ensure efficient resource utilization across the edge-cloud continuum. The research also highlights the role of predictive analytics in anticipating data movement needs, along with real-world applications where AI has successfully been implemented. Furthermore, challenges such as system complexity, computational overhead, and data privacy concerns are addressed. This paper concludes with potential future directions for leveraging AI in distributed file systems, paving the way for more robust and scalable data management solutions in the era of edge and cloud computing.
Downloads
References
Zhang, X., & Li, Y. (2021). AI-driven dynamic data placement in distributed cloud storage systems. Journal of Cloud Computing: Advances, Systems, and Applications, 9(3), 43-56.
Shen, X., & Liu, W. (2020). Machine learning-based data placement for cloud storage optimization. Cloud Computing and Distributed Systems, 12(4), 135-148.
Chen, J., & Wang, L. (2022). Reinforcement learning for optimal data placement in edge computing. IEEE Transactions on Cloud Computing, 10(2), 321-334.
S. Kumari, “Kanban-Driven Digital Transformation for Cloud-Based Platforms: Leveraging AI to Optimize Resource Allocation, Task Prioritization, and Workflow Automation”, J. of Artificial Int. Research and App., vol. 1, no. 1, pp. 568–586, Jan. 2021
Singu, Santosh Kumar. "Designing scalable data engineering pipelines using Azure and Databricks." ESP Journal of Engineering & Technology Advancements 1.2 (2021): 176-187.
Madupati, Bhanuprakash. "Blockchain in Day-to-Day Life: Transformative Applications and Implementation." Available at SSRN 5118207 (2021).
S. Kumari, “Digital Transformation Frameworks for Legacy Enterprises: Integrating AI and Cloud Computing to Revolutionize Business Models and Operational Efficiency ”, Journal of AI-Assisted Scientific Discovery, vol. 1, no. 1, pp. 186–204, Jan. 2021
Singu, Santosh Kumar. "Real-Time Data Integration: Tools, Techniques, and Best Practices." ESP Journal of Engineering & Technology Advancements 1.1 (2021): 158-172.
S. Kumari, “Kanban and AI for Efficient Digital Transformation: Optimizing Process Automation, Task Management, and Cross-Departmental Collaboration in Agile Enterprises”, Blockchain Tech. & Distributed Sys., vol. 1, no. 1, pp. 39–56, Mar. 2021
Madupati, Bhanuprakash. "Kubernetes: Advanced Deployment Strategies-* Technical Perspective." (2021).
Kumar, P., & Singh, R. (2019). Data placement and optimization using reinforcement learning in distributed systems. Proceedings of the International Conference on Edge Computing and AI, 35-42.
Wang, Z., & Tan, J. (2021). Deep learning techniques for dynamic data placement in distributed file systems. Journal of Computing and Information Technology, 29(2), 85-99.
Goh, H., & Lee, J. (2023). Optimizing data placement in edge-cloud environments using AI techniques. International Journal of Cloud Computing and Big Data, 15(1), 72-85.
Martin, P., & Bell, S. (2020). AI in content delivery networks: Optimizing data placement for minimal latency. Networking and Distributed Systems Journal, 18(4), 97-111.
Xu, W., & Chen, Y. (2022). AI-driven data placement for cloud storage systems. International Journal of Cloud Systems, 13(3), 78-91.
Lin, J., & Li, F. (2021). Optimizing data placement in edge-cloud systems: A machine learning approach. Journal of Edge Computing, 6(2), 56-68.
Zhang, X., & Liu, Z. (2023). Reinforcement learning for intelligent data placement in distributed file systems. Journal of Computing and Network Systems, 45(5), 23-34.
Wang, H., & Xu, Z. (2020). The complexity of AI models in distributed environments. International Conference on Distributed Systems, 67-72.
Kumar, S., & Jain, R. (2021). Computational overhead of AI algorithms in edge-cloud computing. Journal of Edge Computing and AI, 13(1), 23-34.
Li, W., & Zhang, T. (2022). Data privacy concerns in AI-driven distributed systems. International Journal of Cloud Security, 12(4), 45-57.
Singh, P., & Raj, S. (2023). Federated learning for privacy-preserving AI in distributed file systems. IEEE Transactions on Distributed Computing, 28(3), 78-89.
