Job Details

Senior Scala Engineer for Spark Platform

Job Info:

Category: Development, Infrastructure
Company Description: Leading global provider of data, news and analytics
Salary: Highly Competitive, Depending on Experience
Position Type: Permanent
Location: San Francisco, CA
Job Number: 9098

Job Description:


The Spark Platform team is building low-latency, distributed analytics infrastructure for the entire firm based on Apache Spark.

Instead of building dozens of isolated Spark applications for individual problem domains, we are building a platform that makes it easy for teams to plug in their business logic without duplicating common functionality. This includes connecting to various datastores or real-time streams, figuring out a way to serverize Spark transforms or having to re-implement transforms such as currency conversion that are very common in financial analytics. But we can't just use Apache Spark as is. We need to enhance open source Spark to fit our low-latency, high throughput and high availability contexts. That's where you come in.

As part of our team, you'll collaborate with a number of the core Spark contributors to co-design and co-develop enhancements to Spark in areas ranging from Performance to HA.

We'll trust you to:

  •  Contribute to the development of the company's Spark Platform
  •  Interact with application teams to understand their specific requirements and drive the next set of features for the Spark Platform
  •  Meet with key players in the open source community to motivate necessary development efforts
  •  Contribute code to the Apache Spark community in areas ranging from Machine Learning to Spark HA

You need to have:

  •  Strong programming proficiency in Java and Scala (a minimum of two years)
  •  An expert-level understanding of Apache Spark - your work will involve modifying its internals

We'd love to see:

  •  Experience with MLlib
  •  Open source contributions of any sort
  •  Professional experience developing analytics infrastructure



All qualified candidates are encouraged to apply by submitting their resume as an MS word document including a cover letter with a summary of relevant qualifications, highlighting clearly any special or relevant experience.

Please Note: All inquiries will be treated with the utmost confidentiality. Your resume will not be submitted to any client company without your prior knowledge and consent.

Contact Recruiter
Senior Technical Recruiter
Andiamo Partners | 90 Broad Street, Suite 1501, New York, NY 10004

Share Share this Job