High-Throughput Data Processing Architectures on AWS for Logistics and Transportation Systems
DOI:
https://doi.org/10.63282/3050-9246.IJETCSIT-V7I1P144Keywords:
AWS, High-Throughput Processing, Logistics Systems, Stream Processing, Batch Processing, Amazon Kinesis, Scalability, Iot Data ProcessingAbstract
Modern logistics and transportation systems generate massive volumes of data from IoT sensors, GPS trackers, fleet management systems, and supply chain operations. This White paper presents comprehensive architectural patterns for implementing high-throughput data processing systems on Amazon Web Services (AWS) that handle both streaming and batch workloads. The proposed architectures leverage AWS services, including Amazon Kinesis Data Streams, AWS Lambda, and Amazon EMR to provide scalable, reliable, and cost-effective solutions. Key focus areas include real-time streaming processing for operational insights, batch processing for historical analysis, and fault tolerance mechanisms. Implementation patterns demonstrate processing capabilities exceeding 50,000 messages per second while maintaining sub-second latency for critical logistics operations.
Downloads
References
[1] “AWS Tools for Batch and Stream Data Transformation”, AWS for Engineers Blog, 2025. [Online] Available: https://awsforengineers.com/blog/aws-tools-for-batch-and-stream-data-transformation/
[2] “Data Pipelines on AWS: 12 Services & How to Use Them Effectively”, Dagster Guides, 2024. [Online]. Available: https://dagster.io/guides/data-pipelines-on-aws-12-services-how-to-use-them-effectively
[3] “Reference architecture”, AWS Well-Architected Framework - Analytics Lens. [Online]. Available: https://docs.aws.amazon.com/wellarchitected/latest/analytics-lens/reference-architecture-2.html
[4] “BUILD A REALTIME DATA PIPELINE SCALABLE APPLICATION DATA ANALYTICS ON AMAZON WEB SERVICES (AWS)”, ResearchGate, 2024. [Online].Available:https://www.researchgate.net/publication/384200712
[5] “Building a 4x faster and more scalable algorithm using AWS Batch for Amazon Logistics”, AWS HPC Blog, 2023. [Online]. Available: https://aws.amazon.com/blogs/hpc/building-a-4x-faster-and-scalable-algorithm-using-aws-batch-for-amazon-logistics/
[6] “Let’s Architect! Designing systems for stream data processing”, AWS Architecture Blog, 2023. [Online].Available:https://aws.amazon.com/blogs/architecture/lets-architect-designing-systems-for-stream-data-processing/
[7] “Building a Modern Data Engineering Architecture on AWS with Databricks”, Medium, 2025. [Online]. Available: https://medium.com/@hamidpmp/building-a-modern-data-engineering-architecture-on-aws-with-databricks-639781d8f4ab
[8] “How AWS Data Analytics Drive Operational Efficiency & Supply Chain Resilience”, SourceFuse Blog, 2025. [Online]. Available: https://www.sourcefuse.com/resources/blog/powering-the-future-of-logistics
[9] “AWS Serverless data analytics pipeline reference architecture”, AWS Big Data Blog, 2020. [Online]. Available:https://aws.amazon.com/blogs/big-data/aws-serverless-data-analytics-pipeline-reference-architecture/
[10] S. K. Sunkara, A. I. Ashirova, Y. Gulora, R. R. Baireddy, T. Tiwari and G. V. Sudha, "AI-Driven Big Data Analytics in Cloud Environments: Applications and Innovations," 2025 World Skills Conference on Universal Data Analytics and Sciences (WorldSUAS), Indore, India, 2025, pp. 1-6, doi: 10.1109/WorldSUAS66815.2025.11199123.
