Project/ Tech Lead – Data Engineering

Qualification

UG: Any Graduate, B.Tech/B.E. in ComputersPG: Any PostgraduateDoctorate: Any Doctorate, Doctorate Not Required

Skills

Spark, Hadoop, Scala, ETL, Data Integration

Employment Type

Permanent

Experience

6-8 Yrs

Salary

Not disclosed

Job description

:- Solid Big-Data Technologies hands on working experience- Strong hands-on experience of programming language like Java/Python/Scala/Spark- Should be very strong in the fields of Hadoop/MapReduce, HDFS, Hive, Hbase, Kafka, NoSQL Databases- Working experience with various ETL techniques and frameworks- Produce ETL design guidelines to ensure a manageable ETL infrastructure- Hands on working experience in analyzing source system data and data flows, working knowledge of structured and unstructured data.- Should be very strong in performance-optimized tuning techniques during data processing using Spark and data storage- Hands on working experience of data processing at scale with event driven systems, message queues (Kafka/Flink/ Spark Streaming)- Should possess strong experience in the execution of projects with the test-driven development approach in the Agile DevOps environment.- Should possess strong experience in handling Agile/Scrum based Data Engineering teamworking in a development environment designing and maintaining data stores- Proficient understanding of distributed computing principles- Hands on working experience on any of the data engineering/ analytics platform (Hortonworks/ Cloudera/ MapR), Hortonworks preferred- Experience building data pipelines for structured/ unstructured, real-time/batch, events/synchronous/asynchronous using Kafka, Steam processing- Experience gathering and processing raw data at scale (including writing scripts, calling APIs, write SQL queries, etc.)- Strong technical, analytical, and problem-solving skills- Excellent written and verbal communication skills