It’s easy for a company to talk itself out of using MLOps. After all, you have to integrate it into your data science, engineering and IT teams, which increases the onboarding process and requires your current employees to learn all new skills and practices. It will probably require a top-down rework of many of their workflows. With the up-front effort being so high, is it really worth it?
Absolutely.
Like its traditional software development cousin DevOps, MLOps makes the essential parts of machine learning (ML) — data management, algorithm design and deployment — flexible and scalable. In addition, the system’s inherently collaborative nature will improve overall communication between data scientists, DevOps engineers and IT engineers. And while the initial time and energy cost is high, MLOps practices improve long-term efficiency by creating more universal workflows and allowing teams to spend less time on individualized project work.
Overall, making the transition to MLOps practices can benefit your team’s workflow, output and collaborative ability. To learn more, Built In NYC sat down with leadership at CompStak, a database for commercial real estate, and VideoAmp, a media measurement company focused on the advertising industry. We talked about the reasons and challenges of MLOps implementation — and why it has been well worth it in the long run.
Why did your teams adopt MLOps to transition your work into production?
CompStak’s product is data-centric and our ML service has direct impacts on client-facing services. We have various types of ML projects including record linkage, computer vision, named entity recognition (NER) and time series models. We adopted modern MLOps practices to ensure data science teams can easily perform experiments in an isolated environment and perform integration tests without having to set up the entire ecosystem. On the development side, we have implemented a continuous integration in which an ephemeral environment is spun up and triggered by a Github pull request. On the deployment side, embracing MLOps practices has empowered data scientists and ML engineers to easily collaborate on scheduled training and automatic model redeployment, building the foundation for continuous learning projects.
What are the main barriers to entry for teams looking to adopt MLOps practices, and how can they be overcome?
MLOps is an emerging domain within the data science industry, and the adoption requires participation from data analysts, data scientists and ML engineers. Since there is no one-size-fits-all solution and the covered scope is so broad, participants often need to spend time learning the new know-how — it inevitably increases onboarding time for new team members to adapt to the process. That said, that’s also one of the most attractive aspects of data science: Everything is evolving rapidly.
Just like other infrastructures works, the long term value of MLOps can only be harvested after initial investments. Take the monitoring of serving data distribution, for example — it could take months or even years for a feature like that to shine after a ML project is deployed in production. It requires a team to have an innovative culture that values sustainable and scalable ML model deployment, which often can be sacrificed for startup teams due to opportunity costs that seem to be high in the short term.
MLOps adoption empowers data scientists and ML engineers to focus more on contemplating smarter workflows that can be shared across projects.”
How has the nature of your day-to-day work changed since you’ve started implementing MLOps methodologies?
The machine learning model is often criticized as a black box — it’s even true within the data science team’s day-to-day workflow. It’s not unusual that it takes more time to communicate the details of a featurization pipeline than the model training itself. This is where MLOps steps in.
Communication is hard — fortunately, data never gets lost in translation. When the team shares common tools like schema validation and feature stores, team members can communicate naturally through data and model artifacts. By embracing MLOps practices, ML Engineers can trace data and model versioning easily. In the meantime, data scientists can also ensure the featurization pipelines remain intact. No secrets are hidden from the beginning of the model training experiments to the final in-production serving workflow.
Similar to how the DevOps concept changed software development, MLOps adoption empowers data scientists and ML engineers to spend less time building out customized work for individual projects and focus more on contemplating smarter workflows that can be shared across projects.
Why did your teams adopt MLOps to transition your work into production?
MLOps is the commonly accepted name for what we’re doing, but at VideoAmp we think of it as DataOps — it’s a more accurate description. A model is just another transformation of data, and they are only as good as the data that goes into them. Our processes and methodology ensure our commingled dataset is of the highest quality.
What are the main barriers to entry for teams looking to adopt MLOps practices, and how can they be overcome?
The main barriers vary depending on the team’s starting point and company dynamics. VideoAmp is an agile organization with incredibly talented and experienced engineers who quickly design and develop complicated systems. One of the barriers we faced was establishing guardrails and enforcing design and development consistency. Putting these frameworks in place ensured we set junior developers up for success.
With a growing team, we can now better balance the resources and hours spent on herculean efforts and leverage our team’s strengths in more efficient ways.”
How has the nature of your day-to-day work changed since you’ve started implementing MLOps methodologies?
We’re still in the process of perfecting our development frameworks. I’m currently focusing on redistributing workloads across our team, allowing senior engineers to spend more of their time designing, distributing and reviewing the work of junior developers. With a growing team, we can now better balance the resources and hours spent on herculean efforts and leverage our team’s strengths in more efficient ways.
