GCP Big Data Engineer – Python, Spark, Dataflow, BigQuery & ETL/ELT

Job Summary Synechron is seeking a highly skilled Data Engineer specializing in Google Cloud Platform (GCP) and Big Data technologies to architect, develop, and optimize scalable data pipelines and data management solutions. This role is pivotal in supporting enterprise data initiatives, ensuring data quality, security, and accessibility for critical business insights. The successful candidate will collaborate across teams to drive data platform innovations and operational excellence aligned with organizational goals. Software Requirements Required:Extensive experience with GCP services including BigQuery, Dataflow, Cloud Storage, and Cloud Pub/Sub Strong proficiency in Apache Spark for distributed data processing and analytics Hands-on expertise in building and maintaining data pipelines using ETL/ELT processes Proficiency in Python for data scripting, automation, and orchestration tasks Experience with distributed data storage and management, including relational and NoSQL databases (PostgreSQL, MySQL, MongoDB) Familiarity with version control tools such as Git Knowledge of Linux/Unix environments for data processing and scripting Preferred:Experience with data governance, metadata management, and data security best practices Knowledge of data orchestration tools like Apache Airflow or Prefect Understanding of containerization (Docker) and orchestration (Kubernetes) for data deployment Overall Responsibilities Design, develop, and maintain scalable, resilient data pipelines on GCP to support enterprise analytics and reporting solutions Collaborate with business stakeholders, data scientists, and analytics teams to understand data requirements and implement optimized data workflows Implement and enforce data quality, security, and governance standards across platforms Optimize data ingestion, transformation, and processing workflows to ensure high performance and cost efficiency Perform data profiling, troubleshooting, and resolution of pipeline issues to ensure operational reliability Stay current with emerging data engineering best practices, tools, and industry trends, leading continuous improvement initiatives Technical Skills (By Category) Programming Languages: Required: Python for scripting and orchestration Preferred: SQL, Java (for integration or data processing tasks) Databases & Data Management: BigQuery, PostgreSQL, MySQL, MongoDB, data modeling, data security, and query optimization Cloud Technologies: GCP services including BigQuery, Dataflow, Cloud Storage, Pub/Sub, IAM, and Cloud Function deployment Frameworks & Libraries: Apache Spark, Dataflow, Airflow (preferred), TensorFlow or PyTorch (if ML integrations are involved) Development & Orchestration Tools: Git, Jenkins, Docker, Kubernetes, Terraform, Apache Airflow, and other CI/CD tools Security & Compliance: Implementing security policies, data encryption, access controls, and compliance standards such as GDPR or HIPAA Experience Requirements