Big Data Engineer – Java, Apache Spark, AWS EMR, Lambda, EKS & Airflow

Job Summary Synechron is seeking a Big Data Engineer with 5+ years of experience in designing, developing and deploying scalable and reliable data-processing solutions on AWS using Java and Apache Spark. The role will focus on implementing big data processing pipelines using Apache Spark on AWS EMR, developing and deploying serverless applications using AWS Lambda, utilizing Amazon EKS for container orchestration and microservices management, and designing workflow orchestration using Apache Airflow. The position will contribute to business objectives by delivering reliable, scalable and maintainable data-processing solutions that support efficient data operations and application services. Software Requirements Required Java: Strong hands-on experience in Java development. Apache Spark: Proficiency in Apache Spark for distributed data processing. AWS EMR: Experience implementing big data processing pipelines using Apache Spark on AWS EMR. AWS Lambda: Experience developing and deploying serverless applications using AWS Lambda. AWS Service Integration: Experience integrating AWS Lambda with other AWS services. Amazon EKS: Experience utilizing Amazon EKS for container orchestration and microservices management. Apache Airflow: Experience designing and implementing workflow orchestration for complex data pipelines. AWS: Experience designing, developing and deploying scalable and reliable data-processing solutions on AWS using Java and Spark. Project-Supported Versions: Ability to work with the versions of Java, Apache Spark, AWS services and related tools supported by the project. Preferred No additional preferred software skills are specified in the requirements. Additional experience relevant to Java, Apache Spark, AWS, AWS EMR, AWS Lambda, Amazon EKS or Apache Airflow may be considered beneficial. Overall Responsibilities Design, develop and deploy scalable and reliable data-processing solutions on AWS using Java and Apache Spark. Implement and maintain big data processing pipelines using Apache Spark on AWS EMR. Develop and deploy serverless applications using AWS Lambda and integrate them with other AWS services. Utilize Amazon EKS for container orchestration and microservices management. Design, implement and maintain Apache Airflow workflows for complex data pipelines. Develop maintainable Java applications and data-processing components aligned with technical requirements. Contribute to solution design, technical discussions and implementation decisions within agreed project standards. Test, troubleshoot and resolve application, pipeline, integration and deployment issues. Support reliable operation of data-processing solutions across relevant development, testing and production environments. Monitor application and pipeline behavior and contribute to improvements in scalability, reliability and maintainability.