Data ops, data engineering, data development — oh my!
From new roles and teams to new skills and processes, the hot topic on everyone’s mind is data ops. I started to notice the data ops emergence back in 2015 as companies began to look at Agile development to spin up new data capabilities rapidly. Later, as data preparation entered the market, ETL developers were gravitating to these tools for quick data loading with transparency into newly formed analytics lakes. Step into today and running advanced analytics (or the sexier term today: machine learning) in real time, and there is a lot of talk about the challenge of moving and updating analytics models from lab environments into production settings.
There is no doubt that vendors such as DataKitchen, DataRobot, and Metis Machine are all messaging and offering workbenches and capabilities to support data ops needs. And there is certainly a lot of gray area in the data platform communities of Informatica and Talend or the data science workbenches such as CognitiveScale that position to help with the engineering and instrumentation of data pipelines and model deployments/refreshes, the goal being a one-click method to push models to production or ease the burden.
Further still, let’s not forget the DevOps workbenches and testing platforms . . .
But is there a market for the data ops workbench? Or are data engineers just more prevalent and equipped with the skills and ingenuity to more quickly help move models to production — using the tools, coding, or both to just get it done?
Vendors, if you are out there: Flood my comments section, inbox, briefing requests, and LinkedIn to show what you’ve got. You may just make it into my Q4 research and evaluation.
Forrester clients: What tools, workbenches, and technologies do your data engineers and developers use to rapidly build and manage quality pipelines and push analytics models to production? Help me set the evaluation criteria for what you want. Shoot me an email!
Okay, I threw down the gauntlet. Who’s ready to leap?