Lead Data Engineer at <intent>
<intent> isn’t your usual company. Our work is powered by the premise that every person at is unique, possessing a distinct set of skills, personality, and passions. We embrace our collective talents to tackle technical challenges, refine our successfully disruptive business ideas, and co-create one of the most human and inspiring work cultures out there. We are a team of collaborators, valuing and rewarding shared success over individual heroics.
As a Staff Data Engineer at <intent>, you’ll code and innovate on products that are changing the e-commerce industry. Our engineering teams help the world’s leading travel companies to serve tens of millions of ads every day around the world. These ads provide an exceptional user experience to help shoppers find and compare prices, while simultaneously increasing brand profitability.
In our engineering work, we balance cutting-edge technologies with industry-proven practices. With our polyglot approach, you will see a variety of languages, libraries, and databases. Our agile way of working is founded on many years of experience; you will be working with accomplished innovators in the software community.
We believe that engineers are most effective when they are involved with the whole lifecycle of software development: conceiving new features, designing and coding the change, testing, deploying, and support. Don't worry if you don't know how to do everything at first - you'll learn the essentials through mob & pair programming with other experienced engineers on your team.
This role is for our Data Engineering team that produces, engineers and maintains the <intent> Scores. This team is responsible for doing research in tandem with our data science team, building models everyday and supporting the online serving of our modeling capabilities.
What You Will Do:
- Identify regression, code, feature, and design risks
- Coordinate with team members, including product managers, to help determine the most appropriate priority of stories and tasks based on value, cost, and interdependence
- Review test results to ensure functionality of existing programs and recommend corrective action, consult with development engineers in resolution of problems
- Design, code, and execute automated test scripts and select methods and techniques for obtaining solutions
- Prepare reports and documentation updates, and provide technical consultation on automation issues, tools and processes
- Produce quality code and help improve our code base
- Be a part of a dynamic team and architect solutions to complex problems in the field of data science
<intent> is looking for a bright and ambitious Staff Data Engineer to bring their fresh perspective to our team of thoroughly awesome humans and dogs.
- 7+ years of progressive experience in software development and testing
- Experience working with Scala for 3+ years and strong comfort contributing to a mature Scala code base
- Experience with Apache Spark for 2+ years and an understanding of the patterns behind the engine and you feel comfortable creating, augmenting. debugging, and testing Spark jobs.
- Demonstrated experience with all aspects of delivering working software, including analysis, design, automated testing, continuous integration and continuous deployment
- Proven ability to build and deploy large-scale ML pipelines supporting mission-critical production use cases
- Experience developing, monitoring, and maintaining distributed systems & frameworks (Spark/Hadoop, MapReduce, etc.)
- Proficiency in working on all aspects of delivering working software, including analysis, design, automated testing, CI/CD, operations and support
- Expertise with AWS (AWS Lambda, Kinesis, API Gateway, ECS, Redshift, etc)
- Bachelor’s degree in computer science, software engineering or any other engineering field
- Extra credit if you love functional programming!
<intent> is the data science company for the world’s leading online commerce and travel brands. Our Predictive Intelligence Platform uses patented technology to predict user behavior in real-time and identify the future value of every user. Over 450 innovative brands from more than 40 countries trust Intent’s real-time predictions to deliver personalized user experiences that maximize utility and ROI.
Our team is over 100 people and our offices span globally. We’re headquartered in NYC with locations in London, Kuala Lumpur, and Sao Paulo.
Every day, we’re inspired by two pursuits. First, we’re building novel products that are upending e-commerce. Second, we’re building the company we’ve always wanted to work for — one that’s open, human and collaborative, where very smart people come together to share ideas and get things done. We’re included on Built in NYC's Best Places to Work list and have been on Crain’s 100 Best Places to Work in NYC list for seven years running.
Love Your Job:
Our employees enjoy coming to work, and we let them know they're valued.
Our vibrant team accomplishes a lot every day, but we insist upon work/life balance so things never become stale. We don’t take ourselves too seriously, but we take our work very seriously.
We believe that in order for our employees to perform their best, they need access to strategic decisions, and so our flat structure and open communication invite innovation from all levels — ideas flow freely.
We offer competitive compensation, stock options, and great perks & benefits, including:
- Flexible paid time off
- A generous parental leave policy
- A beautiful, dog-friendly office in SoHo with drinks and snacks
- An open environment with lots of natural light
- Annual $2,000 learning budget and Citi Bike membership
- Access to Fond, our employee perks program featuring deals and discounts on hundreds of products and services
- Access to Sherpaa, a telehealth service with 24/7 support
- Company-wide social events, and more!
So what are you waiting for? Apply with your resume in just a few clicks!