Req ID: 98730
NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.
We are currently seeking a Spark Hadoop Data Engineer to join our team in Irving, Texas (US-TX), United States (US).
Job Duties and Responsibilities:
- Analyze and understand data sources & APIs
- Design and Develop methods to connect & collect data from different data sources
- Design and Develop methods to filter/cleanse the data
- Design and Develop SQL, Hive queries, APIs to extract data from the store
- Work closely with data Scientists to ensure the source data is aggregated and cleansed
- Work with product managers to understand the business objectives
- Work with cloud and data architects to define robust architecture in cloud setup pipelines and work flows
- Work with DevOps to build automated data pipelines
- Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Yarn, Hive, Pig, Impala, Spark, Kafka,)
- Good understanding for ETL tools like Ab-initio, TalenD
- Familiarity with HTTP and invoking web-APIs
- Exposure to machine learning engineering
- Exposure to NLP and text processing
- Experienced in managing work with distributed teams
- Experience working in SCRUM methodology
- Proven sense of high accountability and self-drive to take on and see through big challenges
- Confident, takes ownership, willingness to get the job done
- Excellent verbal communications and cross group collaboration skills
- 3+ years of Advanced knowledge of Hadoop ecosystem and Big Data technologies
- 3+ years of experience and Expert level knowledge building pipelines using Spark/Pyspark
- 3+ years of experience in programming in Scala and Python
About NTT DATA Services
NTT DATA Services is a global business and IT services provider specializing in digital, cloud and automation across a comprehensive portfolio of consulting, applications, infrastructure and business process services. We are part of the NTT family of companies, a partner to 85 % of the Fortune 100.
NTT DATA Services is an equal opportunity employer and will consider all qualified applicants for employment without regard to race, gender, disability, age, veteran-status, sexual orientation, gender identity, or any other class protected by law.
Spark Hadoop Data Engineer – Irving, Texas