Data Engineering Lead

Experience
7 to 10 Years

Mandatory Skills

Data Architecture
Leadership
Matillion
DBT
SQL
NoSQL
Python
Snwoflake
Azure/AWS
Additional Skills

Communication skills
Stakeholder management
Job Description

About Us Logic

Pursuits provides companies with innovative technology solutions for everyday business problems. Our passion is to help clients become intelligent, information-driven organizations, where fact-based decision-making is embedded into daily operations, which leads to better processes and outcomes. Our team combines strategic consulting services with growth-enabling technologies to evaluate risk, manage data, and leverage AI and automated processes more effectively. With deep, big four consulting experience in business transformation and efficient processes, Logic Pursuits is a game-changer in any operations strategy.

Job Description

We are looking for an accomplished and dynamic Data Engineering Lead to join our team and drive the design, development, and delivery of cutting-edge data solutions. This role requires a balance of strong technical expertise, strategic leadership, and a consulting mindset. As the Lead Data Engineer, you will oversee the design and development of robust data pipelines and systems, manage and mentor a team of 5 to 7 engineers, and play a critical role in architecting innovative solutions tailored to client needs.

You will lead by example, fostering a culture of accountability, ownership, and continuous improvement while delivering impactful, scalable data solutions in a fast-paced, consulting environment.

Key Responsibilities

Client Collaboration:

Act as the primary point of contact for US-based clients, ensuring alignment on project goals, timelines, and deliverables.
Engage with stakeholders to understand requirements and ensure alignment throughout the project lifecycle.
Present technical concepts and designs to both technical and non-technical audiences.
Communicate effectively with stakeholders to ensure alignment on project goals, timelines, and deliverables.
Set realistic expectations with clients and proactively address concerns or risks.

Data Solution Design and Development:

Architect, design, and implement end-to-end data pipelines and systems that handle large-scale, complex datasets.
Ensure optimal system architecture for performance, scalability, and reliability.
Evaluate and integrate new technologies to enhance existing solutions.
Implement best practices in ETL/ELT processes, data integration, and data warehousing.

Project Leadership and Delivery:

Lead technical project execution, ensuring timelines and deliverables are met with high quality.
Collaborate with cross-functional teams to align business goals with technical solutions.
Act as the primary point of contact for clients, translating business requirements into actionable technical strategies.

Team Leadership and Development:

Manage, mentor, and grow a team of 5 to 7 data engineers; Ensure timely follow-ups on action items and maintain seamless communication across time zones.
Conduct code reviews, validations, and provide feedback to ensure adherence to technical standards.
Provide technical guidance and foster an environment of continuous learning, innovation, and collaboration.
Support collaboration and alignment between the client and delivery teams.

Optimization and Performance Tuning:

Be hands-on in developing, testing, and documenting data pipelines and solutions as needed.
Analyze and optimize existing data workflows for performance and cost-efficiency.
Troubleshoot and resolve complex technical issues within data systems.

Adaptability and Innovation:

Embrace a consulting mindset with the ability to quickly learn and adopt new tools, technologies, and frameworks.
Identify opportunities for innovation and implement cutting-edge technologies in data engineering.
Exhibit a "figure it out" attitude, taking ownership and accountability for challenges and solutions.

Learning and Adaptability:

Stay updated with emerging data technologies, frameworks, and tools.
Actively explore and integrate new technologies to improve existing workflows and solutions.

Internal Initiatives and Eminence Building:

Drive internal initiatives to improve processes, frameworks, and methodologies.
Contribute to the organization’s eminence by developing thought leadership, sharing best practices, and participating in knowledge-sharing activities.

Qualifications Education:

Bachelor’s or master’s degree in computer science, Data Engineering, or a related field.
Certifications in cloud platforms such as Snowflake Snowpro, Data Engineer is a plus.

Experience:

8+ years of experience in data engineering with hands-on expertise in data pipeline development, architecture, and system optimization
Demonstrated success in managing global teams, especially across US and India time zones.
Proven track record in leading data engineering teams and managing end-to-end project delivery.
Strong background in data warehousing and familiarity with tools such as Matillion, dbt, Striim, etc.

Technical Skills:

Lead the design, development, and deployment of scalable data architectures, pipelines, and processes tailored to client needs
Expertise in programming languages such as Python, Scala, or Java.
Proficiency in designing and delivering data pipelines in Cloud Data Warehouses (e.g., Snowflake, Redshift), using various ETL/ELT tools such as Matillion, dbt, Striim, etc.
Solid understanding of database systems (relational and NoSQL) and data modeling techniques.
Hands-on experience of 2+ years in designing and developing data integration solutions using Matillion and/or dbt.
Strong knowledge of data engineering and integration frameworks.
Expertise in architecting data solutions.
Successfully implemented at least two end-to-end projects with multiple transformation layers.
Good grasp of coding standards, with the ability to define standards and testing strategies for projects.
Proficiency in working with cloud platforms (AWS, Azure, GCP) and associated data services.
Enthusiastic about working in Agile methodology.
Possess a comprehensive understanding of the DevOps process including GitHub integration and CI/CD pipelines.

Soft Skills:

Exceptional problem-solving and analytical skills.
Strong communication and interpersonal skills to manage client relationships and team dynamics.
Ability to thrive in a consulting environment, quickly adapting to new challenges and domains.
Ability to handle ambiguity and proactively take ownership of challenges.
Demonstrated accountability, ownership, and a proactive approach to solving problems.

Additional Information

Why Join Us?

• Be at the forefront of data innovation and lead impactful projects.

• Work with a collaborative and forward-thinking team.

• Opportunity to mentor and develop talent in the data engineering space.

• Competitive compensation and benefits package.

Required Qualification
Bachelor of Engineering - Bachelor of Technology (B.E./B.Tech.) ,

Job Insights: Important Tips to source better

Please look for early joiners. (Max. 30 days)
This is 5 days work from office role (No Hybrid/ Remote opportunities)
We are open to consider relocation

We are looking for candidates with strong experience in data architecture


Potential companies: Tiger Analytics, Tredence, Quantiphi, Data Engineering Group within Infosys/TCS/Cognizant, Deloitte Consulting etc.
Questionnaire

Question1 : Number of team members handling? ?

Desired answer : Minimum 5 members

Question2 : Proficiency in working with cloud platforms (AWS, Azure, GCP) ?

Desired answer : Please specify cloud name

Question3 : Experience in designing and developing data integration solutions using Matillion and/or dbt ?

Desired answer : Minimum 2 years

Question4 : Experience in SQL, NoSQL and Data Modelling? (Please mention separately) ?

Desired answer : Good experience in all

Question5 : Experience in Snowflake and Matillion / dbt? (Please mention separately) ?

Desired answer : Expert level

Question6 : Experience in Python programming languages ? (in years) ?

Desired answer : Expert level

Question7 : Experience in Design, Development and Deployment of Data Architecture? (in years) ?

Desired answer : Minimum 5 years