Software Engineer, Data Orchestration at Stripe

Ideal for a senior engineer with over eight years of professional experience building large-scale data infrastructure. You have a deep understanding of distribu

Work type: unknown

Location: N/A

Type: Full-time

Summary

Ideal for a senior engineer with over eight years of professional experience building large-scale data infrastructure. You have a deep understanding of distributed systems and a passion for creating ergonomic APIs that empower other engineering teams to process petabytes of data reliably. **What makes it worth a look...** Stripe operates massive batch processing infrastructure moving terabytes of data every day, and this role puts you at the center of that scale. You will contribute to open-source technologies like Apache Airflow and Iceberg while designing products that support critical financial operations. **You might be a good fit if you...** - Have 8+ years of experience writing production-level code for data systems. - Are deeply familiar with Spark, Flink, Airflow, and distributed system architectures. - Possess strong API design skills and a focus on developer experience. - Enjoy contributing to open-source software and debugging complex data pipelines.

Job Description

## Who we are

### About Stripe

Stripe is a financial infrastructure platform for businesses. Millions of companies—from the world's largest enterprises to the most ambitious startups—use Stripe to accept payments, grow their revenue, and accelerate new business opportunities. Our mission is to increase the GDP of the internet, and we have a staggering amount of work ahead. That means you have an unprecedented opportunity to put the global economy within everyone's reach while doing the most important work of your career.

### About the team

The Big Data Infrastructure operates the critical infrastructure that powers the batch data processing at Stripe. The team supports a variety of use cases, including Payment, Ledger, ML, Fraud Detection, Product Analytics, Regulatory Reporting, Financial Data Reconciliation, and externally facing products like Radar and Sigma. As an example of the scale, the team's systems serve hundreds of teams, thousands of workflows, 100,000+ task executions, O(billion) transformations, and moving terabytes of data processing over 1 GB/second every day. Our users inside Stripe include other engineering teams, Data Scientists, Sales and Operations, Finance, etc.

Data Orchestration builds and operates the time-based and event-based orchestration infrastructure that powers and accelerates batch data pipelines. The team operates on a wide range of tech stacks including Airflow, Spark, SQL, Kafka, Flink, Hive MetaStore, Trino, Pinot, Python, Java, Scala, S3, and Iceberg.

## What you'll do

As a Software Engineer on this team, you'll design and build infrastructure that powers batch data processing at Stripe.

### Responsibilities

Design, build, and maintain next-generation and first-generation versions of key Data Platform products, with an emphasis on usability, reliability, security, and efficiency.

Design ergonomic APIs and abstractions that build a great customer experience for internal Stripes, that will in turn enhance the experience of millions of Stripe users.

Ensure operational excellence and enable a highly available and reliable Data Orchestration platform across batch workloads.

Collaborate with high-visibility teams and their stakeholders to support their key initiatives—while building a robust platform that benefits all of Stripe in the long term.

Plan for the growth of Stripe infrastructure by unblocking, supporting, and communicating with internal partners to achieve results.

Connect your work with improvements in the usability and reliability of open source software (OSS) like Apache Airflow, Iceberg, and Spark, and contribute back to the OSS community.

## Who you are

We're looking for someone who meets the minimum requirements to be considered for the role. If you meet these requirements, you are encouraged to apply. The preferred qualifications are a bonus, not a requirement.

### Minimum requirements

8+ years of professional experience writing high-quality, production-level code or software programs and an interest in data infrastructure.

Has experience operating or enabling large-scale, high-availability data pipelines from design to execution and safe change management. Expertise in Spark, Flink, Airflow, Python, Java, SQL, and API design is a plus.

Has experience developing, maintaining, and debugging distributed systems built with open source tools.

Has experience building infrastructure-as-a-product with a strong focus on user needs.

Has strong collaboration and communication skills, and can interact with both technical and non-technical participants.

Has the curiosity to learn about new technologies and business processes.

Is energized by delivering effective, user-first solutions through creative problem-solving and collaboration.

### Preferred qualifications

Has experience writing production-level code in Scala, Spark, Flink, Airflow, Python, Java, and SQL.

Has experience designing APIs or building developer platforms.

Has experience optimizing the end-to-end performance of distributed systems.

Has experience scaling distributed systems in a fast-moving environment.

Has experience working with Airflow infrastructure.

Genuine enjoyment of innovation and an interest in understanding how things work.

Office-assigned Stripes spend at least 50% of the time in a given month in their local office or with users. This hits a balance between bringing people together for in-person collaboration and learning from each other, while supporting flexibility about how to do this in a way that makes sense for individuals and their teams.

Office location—Bengaluru, KA, India

View this job on nocollar jobs