Job Title:Infrastructure Engineer.
Location: Bangalore.
Experience: 5 to 8 Years
Must Skills: DevOps, AWS Glue, Hive, Athena, Spark, & Python.
Job Reference ID:BT/F48/IN
Job Description
As a key member of Data Infrastructure team, you will be integral in envisioning, implementing, and evangelizing new features, pipelines, and platforms that not only delivery data, but insight. You will be at the forefront of assessing the most impactful data needs throughout the business and leading the charge in identifying automated solutions that help bring data to life. You have experience building and maintaining ETLs, data pipelines, and data models. Beyond your strong technical skills, you are a collaborator who is a passionate advocate for empowering and enlightening others with the power of data to help turn information into knowledge and actionable insights.
Responsibilities:
➢ Drive and participate in the definition, the direction, and the development of the Data platform.
➢ Work with internal stakeholders, data engineers and data science team to scope the requirements, evaluate business needs and objectives.
➢ Design and develop highly scalable, reliable, extensive, and maintainable Data Processing Frameworks.
➢ Establish and implement best engineering practices such as architectural design, unit and regression testing, test-driven development, pair programming, and continuous integration frameworks.
➢ Assist with process improvement with a customer focused, progressive mindset.
➢ Understand data classification and adhere to the information protection and privacy restrictions on data.
➢ Proficient in containerization technologies, including Docker, and container orchestration with Kubernetes.
➢ Experience with Terraform, CloudFormation and CDK.
➢ Experience with Building CI/CD Frameworks using Jenkins, GitHub Actions for Automated Deployments.
➢ Maintain Git lab Repositories, Handling Releases and Branching activities.
Experience / Qualification:
➢ 5+ years as a professional Database Infrastructure engineer with experience in production scale platforms
➢ Strong experience with distributed data architectures and big data processing technologies (Spark, EMR, Hadoop, Hive).
➢ Strong experience with Configuration Management, Continues integration and DevOps.
➢ Expert knowledge of AWS Analysis services like Redshift, Glue, Athena, Lake Formation.
➢ Strong knowledge on Python or similar scripting language.
➢ Strong experience with Realtime Frameworks, collection, and processing Realtime data using kinesis Data Streams, KCL, KDA and Firehoses.
➢ Strong understanding of AWS security services including KMS, CloudTrail, STS.
➢ Strong understanding of Kinesis/Glue frameworks and Spark Streaming for building Realtime analytics.
➢ Expert in CICD framework, Preferably GitHubActions, Jenkins and GitHub for version controlling.
➢ Expert knowledge on Blue/Green Deployment for Lambda and Microservices Expert Knowledge on Elastic Containers and Elastic Container Services.
➢ Discerning appreciation of best practices in designing data lakes/lake houses, mentoring junior members, peer review and recommendation.
➢ Proven ability to deliver high quality, production ready code.
➢ Expert understanding of the agile software development life cycle.
➢ Strong customer service and communication skills (verbal and written) to effectively interact with a diverse team of people across business and engineering.
➢ Bachelor’s degree in computer science.
➢ Experience with Snowflake and Delta Lake technologies is a plus.
Benefits:
➢ Competitive salary and benefits package.
➢ Opportunity to work on a variety of projects and industries.
➢ Opportunity to work with a team of experienced and talented professionals.
➢ Access to cutting-edge tools and technologies.
➢ Opportunity to grow and develop your skills through ongoing learning and development opportunities
Salary:Â Commensurate with experience and demonstrated competence Contact: hr@bigtappanalytics.com
Contact:Â Â Â Â hr@bigtappanalytics.com