Sr. ML Engineer

This is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to financial services, autonomous cars, Government, academia, research and manufacturing.

"DDN's A3I solutions are transforming the landscape of AI infrastructure." – IDC 

“The real differentiator is DDN. I never hesitate to recommend DDN. DDN is the de facto name for AI Storage in high performance environments” - Marc Hamilton, VP, Solutions Architecture & Engineering | NVIDIA 

DDN is the global leader in AI and multi-cloud data management at scale. Our cutting-edge data intelligence platform is designed to accelerate AI workloads, enabling organizations to extract maximum value from their data. With a proven track record of performance, reliability, and scalability, DDN empowers businesses to tackle the most challenging AI and data-intensive workloads with confidence. 

Our success is driven by our unwavering commitment to innovation, customer-centricity, and a team of passionate professionals who bring their expertise and dedication to every project. This is a chance to make a significant impact at a company that is shaping the future of AI and data management. 

Our commitment to innovation, customer success, and market leadership makes this an exciting and rewarding role for a driven professional looking to make a lasting impact in the world of AI and data storage. 

We are seeking a talented and experienced Senior Machine Learning Engineer to help us deploy AI/ML training and advanced Retrieval-Augmented Generation (RAG) pipelines for high-performance AI applications. You will be responsible for designing, deploying, and optimizing large-scale AI training and inference pipelines. You will work closely with data scientists and software developers to operationalize models using open-source tools like Apache Spark, Airflow, and MLflow. You will collaborate in our efforts to scale Retrieval-Augmented Generation (RAG) pipelines for AI applications, ensuring robust and efficient deployment.

Key Responsibilities:

• Design and deploy large-scale AI/ML training pipelines using open-source tools such as Apache Spark and Apache Airflow.
• Integrate MLflow with DDN’s Infinia product for tracking and managing machine learning experiments, model versioning, and deployment.
• Implement and scale Retrieval-Augmented Generation (RAG) pipelines to enable efficient retrieval of knowledge for generative models.
• Automate, monitor, and optimize the end-to-end ML workflows and pipelines for production-grade applications.
• Work collaboratively with cross-functional teams including data science, engineering, and product to operationalize AI/ML models.
• Maintain and improve CI/CD pipelines for ML models, ensuring smooth transitions from research to production environments.
• Utilize cloud platforms (AWS, GCP, or Azure) for scalable infrastructure management.
• Monitor and troubleshoot pipeline performance issues, implementing solutions to optimize runtime and resource usage.
• Ensure best practices in version control, containerization (Docker, Kubernetes), and infrastructure as code (Terraform, Ansible).
• Keep up-to-date with the latest developments in MLOps, AI/ML frameworks, and tooling.

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Data Science, Machine Learning, or related fields.
• 7+ years of experience in machine learning operations (MLOps) or related roles.
• Extensive experience with Apache Spark, Apache Airflow, and MLflow or equivalent.
• Proven expertise in building and scaling AI/ML pipelines.
• Strong understanding of machine learning frameworks and libraries (TensorFlow, PyTorch, NVIDIA NeMo).
• Experience in deploying open-source vector databases at scale.
• Solid understanding of cloud infrastructure (AWS, GCP, Azure) and distributed computing.
• Proficiency with containerization tools (Docker, Kubernetes) and infrastructure as code.
• Excellent problem-solving and troubleshooting skills, with attention to detail and performance optimization.
• Strong communication and collaboration skills.

Preferred Qualifications:

• Experience with large-scale data processing and storage solutions (Hadoop, Hive, HDFS).
• Knowledge of NLP techniques and tools for model deployment.

Join our dynamic and driven team, where engineering excellence is at the heart of everything we do. We seek individuals who love to challenge themselves and are fueled by curiosity. Here, you'll have the opportunity to work across various areas of the company, thanks to our flat organizational structure that encourages hands-on involvement and direct contributions to our mission. Leadership is earned by those who take initiative and consistently deliver outstanding results, both in their work ethic and deliverables, making strong prioritization skills essential. Additionally, we value strong communication skills in all our engineers and researchers, as they are crucial for the success of our teams and the company as a whole.

Interview Process: After submitting your application, one of our recruiters will review your resume. If your application passes this stage, you will be invited to a 30-minute interview during which a member of our team will ask some basic questions. If you clear the interview, you will enter the main process, which can consist of up to four interviews in total:

• Coding assessment: Often in a language of your choice.
• Systems design: Translate high-level requirements into a scalable, fault-tolerant service (depending on role).
• Real-time problem-solving: Demonstrate practical skills in a live problem-solving session.
• Meet and greet with the wider team.
• Our goal is to finish the main process in 2-3 weeks at most.

DataDirect Networks (DDN) is an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity, gender expression, transgender, sex stereotyping, sexual orientation, national origin, disability, protected Veteran Status, or any other characteristic protected by applicable federal, state, or local law.

#LI-Remote