P03
TransUnion CIBIL · Bengaluru +1 more, India
Free account: apply link, HR contact and an AI-written cold email
TransUnion CIBIL · Bengaluru +1 more, India
Free account: apply link, HR contact and an AI-written cold email
TransUnion's Job Applicant Privacy Notice Team Overview The team is responsible for building, enhancing, and operating scalable data engineering solutions and distributed data processing platforms. The team processes large volumes of data, including terabyte-scale datasets, through recurring batch workloads and production pipelines, with a strong focus on performance, scalability, reliability, and cost efficiency. The role will contribute to developing and optimizing data-intensive applications using Apache Spark, Scala, Python, and cloud technologies, with a preference for Google Cloud Platform (GCP). The team works on identity resolution and data-driven solutions, where linking identities and analyzing relationships across devices, digital identifiers, and other data signals are important. Experience in identity resolution, AdTech, MarTech, or digital identity platforms will be a strong advantage. The team also explores AI/ML concepts and AI-assisted development approaches to improve engineering productivity and enable innovative solutions.

This is a hybrid position and involves regular performance of job responsibilities virtually as well as in-person at an assigned TU office location for a minimum of two days a week. Role Overview And Core Responsibilities * Design, develop, test, and maintain scalable data processing applications using Apache Spark and Scala. * Build and operate production-grade distributed data pipelines capable of processing terabytes of data per execution. * Develop and maintain recurring batch processing workloads, including weekly and scheduled pipelines, ensuring timely completion, reliability, and data quality. * Optimize large-scale Spark jobs for execution time, memory utilization, parallelism, resource consumption, and cloud infrastructure costs. * Design robust data processing workflows with appropriate error handling, retries, checkpointing, recovery mechanisms, and idempotency. * Monitor production jobs, troubleshoot failures and performance bottlenecks, and perform root-cause analysis to ensure operational stability. * Develop cloud-based data engineering solutions, preferably on Google Cloud Platform (GCP), with AWS experience also considered. * Work with technologies such as Google Dataproc, BigQuery, cloud object storage, Parquet, and equivalent cloud data services. * Write clean, efficient, reusable, and maintainable code using Scala, Python, and shell scripting. * Apply strong analytical and problem-solving skills to resolve complex data processing and distributed computing challenges. * Contribute to platform enhancements, automation, workflow orchestration, and continuous improvement of data engineering systems. * Leverage AI/ML concepts and AI-assisted development tools where appropriate to improve development productivity and engineering quality. * Collaborate with engineers, architects, and cross-functional stakeholders to deliver scalable, reliable, and maintainable solutions.
Want to reach TransUnion CIBIL's HR directly?
NextKnock finds the hiring contact and writes a personal cold email from your resume. You send it from your own Gmail.