Work Experience: 10 to 12 Years
Work Location : Bangalore, Chennai & Hyderabad
Must Have Skills: AWS, Glue, DMS, Data integrations and Data Ops.
Job Reference ID:BT/F21/IND
Position Overview:
As the leader of our Data Engineering team, you will be responsible for driving the design, development, deployment and maintenance of our data architecture, pipelines, and processes across both Azure and AWS cloud environments. You will play a critical role in shaping our data strategy, ensuring scalability, reliability, and efficiency while leading a team of talented data engineers.
Key Responsibilities:
Data Strategy and Architecture:
➢ Define and execute a comprehensive data engineering strategy that aligns with the company's overall goals and objectives.
➢ Design and maintain a robust and scalable ETL Framework that could work on both Azure and AWS platforms.
➢ Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
Cloud Platform Expertise:
➢ Demonstrate deep expertise in both Azure and AWS services, leveraging the strengths of each platform to design and implement optimal data solutions.
➢ Familiarity with implementing ETL / ELT using Databricks or Snowflake.
➢ Stay up to date with the latest cloud technologies, tools, and best practices related to data engineering and analytics. Data Pipeline Development:
➢ Lead the design, development, and maintenance of batch and real-time data pipelines to ensure efficient data extraction, transformation, and loading (ETL) process.
➢ Implement scalable frameworks for data quality checks and monitoring mechanisms to ensure the accuracy and reliability of the data.
➢ Establish and enforce data engineering best practices, including coding standards, code reviews, and source code management.
➢ Evaluate new technologies and tools as they continue to evolve and assess their applicability to business goals.
Team Leadership:
➢ Develop and implement the strategic vision for the data engineering team, in line with the company's objectives.
➢ Lead, mentor and grow a team of data engineers, providing guidance, performance feedback, and career development support.
➢ Foster a collaborative and innovative team culture, encouraging continuous learning and knowledge sharing.
Infrastructure Management:
➢ Oversee the deployment, configuration, and management of cloud resources and infrastructure required for data processing and storage.
➢ Optimize resource utilization to ensure cost-effectiveness and high availability
Data Governance and Security:
➢ Implement data governance policies and practices to ensure data compliance, security, and privacy standards are upheld.
➢ Collaborate with security teams to implement appropriate access controls and data encryption mechanisms.
Performance Optimization:
➢ Identify and address performance bottlenecks in data pipelines and processing to ensure optimal query andprocessing speeds.
➢ Proactively monitor system performance and troubleshoot issues as they arise.
Qualifications:
➢ Bachelor’s / master’s degree in computer science, Engineering, or a related field.
➢ Proven experience (10+ years) in data engineering, with a focus on designing and implementing datasolutions on both Azure and AWS platforms.
➢ Strong expertise in cloud-based data storage, processing, and analytics services offered by Azure and AWS
➢ Demonstrated leadership experience, including leading and mentoring technical teams.
➢ Strong expertise in Big Data technologies such as Hadoop, Spark, and Kafka.
➢ Proficiency in SQL and Python for data engineering tasks.
➢ In-depth understanding of data modelling, ETL processes, and data warehousing concepts
➢ Excellent problem-solving skills and the ability to handle complex technical challenges.
➢ Strong communication skills to collaborate effectively with cross-functional teams and present technical concepts to non-technical stakeholders.
➢ Relevant certifications in Azure and AWS are a plus.
Salary: Commensurate with experience and demonstrated competence Contact: hr@bigtappanalytics.com
Contact: hr@bigtappanalytics.com