Custom Software Engineer

Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Python (Programming Language) Good to have skills : Oracle Procedural Language Extensions to SQL (PLSQL), Apache Airflow, Data Build Tool Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary: We are seeking a skilled and motivated Data Engineer to design, develop, and automate scalable data pipelines that support enterprise analytics and business intelligence. The ideal candidate will collaborate with business stakeholders to gather requirements, perform data profiling, implement data quality frameworks, and build robust ELT solutions using Python, SQL, Apache Airflow, DBT Cloud, and Snowflake. This role involves extracting data from multiple source systems, developing automated data pipelines, transforming data using modern data engineering practices, and ensuring high-quality, reliable, and governed data assets. The successful candidate should also have experience with CI/CD, version control, and AI-assisted development practices to improve engineering productivity. Roles & Responsibilities: - Expected to be an SME, collaborate and manage the team to perform. - Responsible for team decisions. - Engage with multiple teams and contribute on key decisions. - Provide solutions to problems for their immediate team and across multiple teams. - Lead the planning and execution of software development activities to meet project goals and timelines. - Collaborate with business stakeholders to gather, analyze, and document business and technical data requirements. - Perform data profiling to understand source data, identify anomalies, and assess data readiness. - Define, implement, and maintain Data Quality (DQ) rules and automated validation checks to ensure data accuracy, completeness, consistency, uniqueness, and integrity. - Design, develop, and automate end-to-end ELT pipelines using Python, SQL, and Apache Airflow. - Extract data from various source systems including REST APIs, relational databases, cloud applications, files, and third-party platforms. - Load, optimize, and manage data in Snowflake while ensuring scalability, performance, and reliability. - Develop reusable data transformation models using DBT Cloud following Medallion Architecture (Bronze, Silver, and Gold layers). - Build metadata-driven and reusable ingestion frameworks with proper logging, monitoring, and error handling. - Validate source-to-target mappings and ensure end-to-end data integrity throughout the pipeline. - Optimize SQL queries, Python code, Airflow workflows, and DBT models for performance and maintainability.