Job Details


Senior Data Engineer

Job Info:


Category: Data
Company Description: The Best Engineered e-commerce Platform in the World
Salary: Highly Competitive, Depending on Experience
Position Type: Permanent
Location:
Job Number: 9857

Job Description:


Job Description

The Data Platforms team at our client is looking for an exceptionally talented Data engineer with an outstanding track record reflecting achievements and skills in modernizing and improving the data infrastructure from the ground up. So, if you are passionate about working with very Large Data sets (structured/unstructured), building large scale Data processing platforms, implementing world class data governance and operational controls, solving complex performance challenges and building robust ETL pipelines then we would love to talk to you.

Our Data Environment

Our Event driven Ecommerce Microservices generate massive volume of data every second. Our Data Platform ingests and organizes this and other third party high speed, high volume data and enables strategic and operational decisions and reporting by Analysts and Data Scientist in every domain of our Ecommerce business.

Our Data Platform supports Strategic and Operational Analytics and leverages Azure hosted Big Data clusters as well as a constellation of Big Microsoft SQL Server instances. Our platform leverages Spark, Hive, Kafka, Redis, Elastic Search, Cosmos DB.   

What you will bring  

·         Degree in Computer Science, Information Technology, Math or related technical field.

·         Natural inclination for designing well thought out data solutions as well as solid hands-on implementation capability

·         5+ years’ experience in engineering data solutions using technologies including Spark (batch/streaming), Scala/Java (building data pipelines vs frameworks), Hadoop, Hive, HBase, Kafka, Spark, Oozie, Yarn.

·         Solid hands-on experience in building data pipelines, deploying and managing Big Data infrastructure, establishing deployment and operational excellence of Big Data clusters.

·         Experience in Relational databases, Data Warehousing, SQL, ETL and/or NOSQL databases will be sweet.

·         Proven experience in Building or improving large scale data infrastructure from the ground up. This could include building Data warehousing stores, Data Architecture, Data Integration, building highly performant data movement pipelines, building tools and automation to facilitate Data and operational governance, Data Lineage, Automation and Monitoring.

·         Proven Performance Tuning Accomplishments and Advanced tuning and trouble shooting skills in Large data environments on the Hadoop or RDBMS stack.

·         Advanced knowledge of internals of at least one RDBMS (MSSQL preferred) and best practices.

·         Data Ops experience including hands on skills in scripting language such as Python, Perl or Bash

·         Proven experience in trouble shooting, problem determination and rapid problem resolution

·         Experience and ability to work under high pressure in a complex technical environment.

 

 

 

What you will Do

 

·         Play a pivotal design and hands on implementation role in improving the Data infrastructure in a project-oriented work environment.

  • Influence cross functional architecture in sprint planning
  • Gather and process raw data at scale - collect data across all business domains (our functional-first, event sourced, micro services backend) and expose mechanisms for large scale parallel processing
  • Design, implement and manage a near real-time ingestion pipeline into a data warehouse and Hadoop data lake.
  • Process unstructured data into a form suitable for analysis and then empower state-of-the-art analysis for analysts, scientists, and APIs
  • Build efficient new Data Models and refactor existing ones. Partner with business to build right data models and analytics capabilities. 

·         Solve complex SQL and Big Data Performance challenges.

·         Mitigate Risks in our data infrastructure by developing the best in class tools and processes.

·         Implement controls, policies, processes and best practices in the Data Engineering space.

  • Evangelize an extremely high standard of code quality, system reliability, and performance.

·         Help us improve our database deployment and change management process.

·         Provide reliable and efficient Data services as part of the database team.

·         Work closely with the devs on development best practices and standards.

·         Be a mentor.


All qualified candidates are encouraged to apply by submitting their resume as an MS word document including a cover letter with a summary of relevant qualifications, highlighting clearly any special or relevant experience.


Please Note: All inquiries will be treated with the utmost confidentiality. Your resume will not be submitted to any client company without your prior knowledge and consent.


Contact Recruiter
arianna.gelwicks@andiamogo.com
Senior Technical Recruiter
Andiamo Partners | 90 Broad Street, Suite 1501, New York, NY 10004


Share Share this Job