When customers experience our DT workshops the common response is, “We didn’t know Microsoft did these things.” Exactly! Data scientists do more than just oil their slide rules and polish their pocket protectors. We actually use science to solve problems in fun ways. Even if you have an army of data scientists it’s likely they could use a little help with DT.
We can always spot customers who have strong data scientists, but who are not trained in Design Thinking. They will build brilliant solutions to problems that may not exist. The MTC can help. Many customers allow us to host their DT sessions so they can learn our methods. We love to do this! And we’ve actually won awards for our DT sessions.
We hear this a lot. Often this is due to the wrong work being assigned to your data scientists. As an example: most younger data scientists are well-versed in software development practices (such as DevOps, git, and scrum). But some are not software developers and struggle when they need to build their own DevOps processes.
Does it really make sense to have a data scientist doing software development (or data engineering) if she isn’t really good at it?
The MTC can help you focus your data scientists and ensure they are actually doing data science. Examples:
automlwith monitoring facilities that will detect model drift and data skew. This service (which is basically “enhanced MLFlow”) is invaluable to data scientists that are not trained in software development. The MTC can help you discover the patterns that work.
Most of the companies I talk to have a data science team. Often they are swamped, working on extracting value from high priority use cases. Often what is needed is something like “Rent-a-Data-Scientist”. But, throwing bodies at a problem, or outsourcing it, rarely works. What we like to do with customers is take on a riskier project and then add value by creating and building processes that are re-usable and will be beneficial to the rest of the team. Some examples:
Companies don’t fail at data science due to a lack of use cases; they fail because they have too many.
The MTC can help you learn and experiment with technology and processes while you stay focused on core use cases.
“We hired smart data scientists, but we don’t know how to get started. Can you help us with a good first use case?”
This is another mantra we hear from customers. Some organizations have tons of possible use cases (see above), others are struggling to come up with their first use case. Data scientists do more than just statistics and neural network building. While these are important, other soft skills like Design Thinking can prove invaluable, especially when you want to find a compelling use case.
The fact is, it’s very difficult to know if a use case is a good use case and how much time should be invested in it. Data science is not typical software engineering where we can discuss the basics of the product and then go and build it. Data science projects are a series of small experiments and deductive reasoning. We need to constantly evaluate when we are done, which could mean “deploy the model to production” or “let’s scrap this project”. The MTC has the calculus that you can leverage.
“Don’t let perfect be the enemy of good enough.” We’ve seen TOO MANY data science projects fail because the team feared putting an imperfect model into production and continued to tweak the model in development. The problem is: training data is never the same as real, production data. Deploy the model as soon as you can and then monitor it. “No model is perfect, but many are useful”.
Even Business Intelligence teams see radical throughput improvements when they begin to think like a data scientist. The time-to-value is shifted to the left and data projects become less risky with a higher success rate.
Pro tip: At the MTC we’ve learned that agile/scrum rarely works for analytics and data science projects. Instead, consider using Lean management principles (fail-fast, MVPs) to control your initiatives at the program level and then kanban to control the day-to-day activities.
“We don’t have experience managing data science projects”
As mentioned above, data science projects are not typical software projects. We have experience managing these projects and can help you.
Here’s one way we can help: assisting you with your outsourced analytics projects.
Many of our customers outsource their data science projects. This could be because their in-house talent is swamped or they want to leverage a consultancy to solve a particularly difficult analytics problem.
The problem is most consultancies want to manage these engagements as fixed scope, fixed fee engagements…much like a traditional software project. But, as I’ve said many times already, data projects are not linear, they are iterative. In many cases we have to “start over” because we had a flaw in our deductive reasoning. With a fixed scope SOW it’s in the consultancy’s best interest to deliver a flawed product that meets the SOW, yet provides no value.
Think about it: how can a consultancy provide a SOW for a data science project that is filled with unknowns? They can’t. The only way this can work is if the contract is structured like a staff augmentation engagement. I’ve written about this extensively.
So, you’ve outsourced your project for a fixed scope SOW to a consultancy and now the project is faltering and not providing the value you thought it would. How can the MTC help? We can conduct a Design Thinking workshop to determine the true problem, evaluate possible solutions, and then determine the best solution using “Cost Impact Analysis” (as well as many other tricks). We can then work with your consultants to transition into truly valuable work. We’ve never failed doing these exercises and the consultancies never mind altering their SOWs if they see the value in delivering the right product to a happy customer.
Yes. Here’s one way: let us host your next hackathon!
Hackathons bring diversity of thought. They are like a combination of Design Thinking and Rapid Prototypes. Not only are you solving business problems but your staff is learning new approaches to analytics and new technologies.
Most companies provide time for their techies to run their own hackathons. While this is a good start, if you truly understand how to run a success hackathon you know it’s more than just a Saturday where your developers write some code. If hackathons are done correctly they are integral to your sprint and product planning. Hackathons should also be a little competitive. Motivation comes from competition. Gamification is a big piece of that. We can help you create hackathons that drive business outcomes yet feel highly creative and fun, while providing the structure to ensure success.
We love hackathons. But they are totally underused in Corporate America. If you are a member of a software development scrum team consider replacing your sprint planning sessions with hackathons. Instead of talking about what we want to solve, why not actually solve something?
Anecdotally, the two biggest reasons data science projects fail:
Yes! We have an entire workshop on this. We think our approach is unique: We don’t just build an innovation lab with you, instead we focus on the processes that are most effective to drive growth. Leveraging the cloud is integral to our approach. Nothing stifles innovation more than having a great business idea that has to wait a month for infrastructure provisioning. We can teach you these processes, you can evaluate them, determine what works for you, and then scale your processes to your entire organization.
“We didn’t know the MTC did that” –a common reaction from our customers
We, of course, focus heavily on our tech solutions at Microsoft, but we also strive to be the Trusted Advisor for our customers. MTC architects have actually done this stuff in the real world and we know the patterns and anti-patterns. A lot of what we do at the MTC is driving culture change. Data and analytics have made revolutionary advances over just the last few years, we can help you leverage these processes to become more data-driven.
SUCCESS for the MTC is solving challenging problems for respected companies and their talented staff. Our typical engagements are 1-3 days. We want to show value quickly. Does that sound like a good investment of your time? The Digital Transformation is here, and we know how to help. How can we help you with your data science initiatives?
Are you convinced your data or cloud project will be a success?
Most companies aren’t. I have lots of experience with these projects. I speak at conferences, host hackathon events, and am a prolific open source contributor. I love helping companies with Data problems. If that sounds like someone you can trust, contact me.
Thanks for reading. If you found this interesting please subscribe to my blog.
Dave Wentzel CONTENT
Data Science Digital Transformation