MTC Data Science-as-a-Service

At the MTC we offer a service for our customers that I call Data Science-as-a-Service. You could also call it Rent-a-Data Scientist. In this article I will show you some ways to leverage this offering.

I get a little bored this time of year. I’m a data scientist (among other things) for the Microsoft Technology Center. The MTC is a service our customers can leverage to learn about Azure, modern data analytics, data science, and a bevy of other topics. One offering that I particularly love is something I call “Data Science-as-a-Service”, or what I formerly called “Rent-a-Data-Scientist”. The fact is, our customers could always use a little help with data science and analytics. We’re here to help and here’s a few ways we can.

Design Thinking

“Does this AI/ML/Data Science use case make sense?”

Customers ask me this question all the time. There is a lot of hype around what AI can do and the MTC is here to help you. One approach to teasing out a good use case is to have a human-centered design thinking workshop. These sessions are a lot of fun. We work together to solve a business problem by utilizing data science. We take the business problem and have a dialog around how we can improve outcomes by showing empathy with the solutions we create. We want you to walk away from these workshops inspired and energized. We help you create a prototype that you can evangelize after the session with your leadership.

When customers experience our DT workshops the common response is, “We didn’t know Microsoft did these things.” Exactly! Data scientists do more than just oil their slide rules and polish their pocket protectors. We actually use science to solve problems in fun ways. Even if you have an army of data scientists it’s likely they could use a little help with DT.

We can always spot customers who have strong data scientists, but who are not trained in Design Thinking. They will build brilliant solutions to problems that may not exist. The MTC can help. Many customers allow us to host their DT sessions so they can learn our methods. We love to do this! And we’ve actually won awards for our DT sessions.

“We aren’t getting the value we thought we’d get from our data scientists”

We hear this a lot. Often this is due to the wrong work being assigned to your data scientists. As an example: most younger data scientists are well-versed in software development practices (such as DevOps, git, and scrum). But some are not software developers and struggle when they need to build their own DevOps processes.

Does it really make sense to have a data scientist doing software development (or data engineering) if she isn’t really good at it?

The MTC can help you focus your data scientists and ensure they are actually doing data science. Examples:

MLOps: This is an off-shoot of DevOps. By automating mundane CI/CD tasks we can allow data scientists to do what they do best…experiment. I have an MLOps workshop that I do with data science teams. It’s about 2 days and we show both the data science team and the Ops team how to work together to show business value. This is my 2nd most popular workshop at the MTC and it’s a lot of fun.
Assist with other aspects of data: Many times data scientists are asked to build data engineering pipelines, perform ETL tasks, or simply acquire data from 3rd parties. Some data scientists are really good at this, and others need a little assistance. We’re here to help.
Learning the latest technologies: Microsoft has Azure Machine Learning Service that helps data scientists improve their velocity by integrating newer technologies like automl with monitoring facilities that will detect model drift and data skew. This service (which is basically “enhanced MLFlow”) is invaluable to data scientists that are not trained in software development. The MTC can help you discover the patterns that work.
“Help us scale our data science practice”

Most of the companies I talk to have a data science team. Often they are swamped, working on extracting value from high priority use cases. Often what is needed is something like “Rent-a-Data-Scientist”. But, throwing bodies at a problem, or outsourcing it, rarely works. What we like to do with customers is take on a riskier project and then add value by creating and building processes that are re-usable and will be beneficial to the rest of the team. Some examples:

building processes that deploy inferencing endpoints to Kubernetes
showing “Citizen Data Scientists” how to be more productive using our patterns
building MLOps pipelines
creating human feedback loops to ensure what we build gets even more valuable over time.

Companies don’t fail at data science due to a lack of use cases; they fail because they have too many.

The MTC can help you learn and experiment with technology and processes while you stay focused on core use cases.

“Help us Think Like a Data Scientist”

“We hired smart data scientists, but we don’t know how to get started. Can you help us with a good first use case?”

This is another mantra we hear from customers. Some organizations have tons of possible use cases (see above), others are struggling to come up with their first use case. Data scientists do more than just statistics and neural network building. While these are important, other soft skills like Design Thinking can prove invaluable, especially when you want to find a compelling use case.

The fact is, it’s very difficult to know if a use case is a good use case and how much time should be invested in it. Data science is not typical software engineering where we can discuss the basics of the product and then go and build it. Data science projects are a series of small experiments and deductive reasoning. We need to constantly evaluate when we are done, which could mean “deploy the model to production” or “let’s scrap this project”. The MTC has the calculus that you can leverage.

“Don’t let perfect be the enemy of good enough.” We’ve seen TOO MANY data science projects fail because the team feared putting an imperfect model into production and continued to tweak the model in development. The problem is: training data is never the same as real, production data. Deploy the model as soon as you can and then monitor it. “No model is perfect, but many are useful”.

Even Business Intelligence teams see radical throughput improvements when they begin to think like a data scientist. The time-to-value is shifted to the left and data projects become less risky with a higher success rate.

Pro tip: At the MTC we’ve learned that agile/scrum rarely works for analytics and data science projects. Instead, consider using Lean management principles (fail-fast, MVPs) to control your initiatives at the program level and then kanban to control the day-to-day activities.
“We don’t have experience managing data science projects”

As mentioned above, data science projects are not typical software projects. We have experience managing these projects and can help you.

Here’s one way we can help: assisting you with your outsourced analytics projects.

Many of our customers outsource their data science projects. This could be because their in-house talent is swamped or they want to leverage a consultancy to solve a particularly difficult analytics problem.

The problem is most consultancies want to manage these engagements as fixed scope, fixed fee engagements…much like a traditional software project. But, as I’ve said many times already, data projects are not linear, they are iterative. In many cases we have to “start over” because we had a flaw in our deductive reasoning. With a fixed scope SOW it’s in the consultancy’s best interest to deliver a flawed product that meets the SOW, yet provides no value.

Ugh!

Think about it: how can a consultancy provide a SOW for a data science project that is filled with unknowns? They can’t. The only way this can work is if the contract is structured like a staff augmentation engagement. I’ve written about this extensively.

So, you’ve outsourced your project for a fixed scope SOW to a consultancy and now the project is faltering and not providing the value you thought it would. How can the MTC help? We can conduct a Design Thinking workshop to determine the true problem, evaluate possible solutions, and then determine the best solution using “Cost Impact Analysis” (as well as many other tricks). We can then work with your consultants to transition into truly valuable work. We’ve never failed doing these exercises and the consultancies never mind altering their SOWs if they see the value in delivering the right product to a happy customer.

“Can you help us get started with a project?”

Yes. Here’s one way: let us host your next hackathon!

Hackathons bring diversity of thought. They are like a combination of Design Thinking and Rapid Prototypes. Not only are you solving business problems but your staff is learning new approaches to analytics and new technologies.

Most companies provide time for their techies to run their own hackathons. While this is a good start, if you truly understand how to run a success hackathon you know it’s more than just a Saturday where your developers write some code. If hackathons are done correctly they are integral to your sprint and product planning. Hackathons should also be a little competitive. Motivation comes from competition. Gamification is a big piece of that. We can help you create hackathons that drive business outcomes yet feel highly creative and fun, while providing the structure to ensure success.

We love hackathons. But they are totally underused in Corporate America. If you are a member of a software development scrum team consider replacing your sprint planning sessions with hackathons. Instead of talking about what we want to solve, why not actually solve something?

“Can you help us save this failing data science initiative?”

Anecdotally, the two biggest reasons data science projects fail:

data acquisition: Ingesting data and preparing it for analytics is difficult. We have patterns that will remove this risk.
feedback loops: Many data scientists struggle designing feedback loops. A feedback loop is how an AI or ML algorithm truly learns. Designing a good feedback loop is an art and I’ll write about it in a future article. We can help you design and code feedback loops which will make your algorithms BETTER over time.

“Can you help us build a data innovation lab?”

Yes! We have an entire workshop on this. We think our approach is unique: We don’t just build an innovation lab with you, instead we focus on the processes that are most effective to drive growth. Leveraging the cloud is integral to our approach. Nothing stifles innovation more than having a great business idea that has to wait a month for infrastructure provisioning. We can teach you these processes, you can evaluate them, determine what works for you, and then scale your processes to your entire organization.

We have free time, how can the MTC help you?

“We didn’t know the MTC did that” –a common reaction from our customers

We, of course, focus heavily on our tech solutions at Microsoft, but we also strive to be the Trusted Advisor for our customers. MTC architects have actually done this stuff in the real world and we know the patterns and anti-patterns. A lot of what we do at the MTC is driving culture change. Data and analytics have made revolutionary advances over just the last few years, we can help you leverage these processes to become more data-driven.

SUCCESS for the MTC is solving challenging problems for respected companies and their talented staff. Our typical engagements are 1-3 days. We want to show value quickly. Does that sound like a good investment of your time? The Digital Transformation is here, and we know how to help. How can we help you with your data science initiatives?

Are you convinced your data or cloud project will be a success?

Most companies aren’t. I have lots of experience with these projects. I speak at conferences, host hackathon events, and am a prolific open source contributor. I love helping companies with Data problems. If that sounds like someone you can trust, contact me.

Thanks for reading. If you found this interesting please subscribe to my blog.

Dave Wentzel 2021-08-18 CONTENT
Data Science Digital Transformation