You are a data scientist and you are excited about a new idea you have for a decision support tool which can help one of the business units in your company. Or you have packaged a particular machine learning algorithm with a customized set of data queries as a prototype application that you use often and would like to share with other data scientists at your organization.
What is the next step? While you are comfortable creating efficient, elegant code to achieve analytics tasks, you know that you are not a professional application developer. This is the time to work with a professional development team – either within your organization or at an external product company. Here are some things to keep in mind to ensure your success.
Breaking Down and Defining Tasks
You will have the most success if you break down the tasks into discrete, clearly defined pieces of work. These pieces of work should be individually demonstrable and testable and are sometimes called user stories. Breaking down a larger piece of software in this way allows you to receive working code faster, allows for you to provide more frequent feedback, and vastly improves communication amongst the whole team – from initial development through to quality control and deployment. Be ready to discuss these requirements frequently if needed. The old school model of throwing huge specification documents “over the wall” is not the most effective approach, especially when working with development teams who utilize an agile or modified/hybrid agile development methodology, as is becoming increasingly common.
You may have an initial set of user stories, but experience shows that these pieces/stories will evolve over time, as the initial pieces are implemented and begin to be used and produce user feedback. This is a key benefit agile software development – more frequent user feedback means less chance of writing code that won’t be used and doesn’t meet the needs of the end user.
The Role of Product/Project Managers
If you are a data scientist at a software company, then chances are you work with (or at least are aware of the existence of) Product Managers. Product Managers spend their days doing a wide variety of tasks – both heads down and heads up – including talking about needs and vision to internal and external customers, reporting status to executive management, writing blog posts(!), collaborating with customer success and product marketing teams, and working closely with development and quality control groups. From company to company, and from day to day, the specific tasks and proportion of time spent on each task vary, but one constant remains – Product Managers are a resource that Data Scientists can utilize to help navigate the course of working with development teams.
Remember how important breaking down tasks into testable, well-defined pieces is? While it is important for everyone in the team to understand the high-level vision, and it is critical to work closely developers on this process, developers are not usually the most efficient resource to own the process of synthesizing the vision into pieces and writing up the details of the breakdown. “Cleverness beans” spent by a developer on absorbing and remembering many details of a potential future state of an application or system means cleverness beans not spent on creating the next key feature. A Product Manager and their team can partner with you on translating your ideas into actionable pieces of development work, well-aligned to the specific development group(s) which will be doing the development.
If you are working with an external software development company rather than one in your own company, then you may find that a Project Manager or Business Analyst is useful to work with in a similar capacity.
Prototyping versus Production
Remember that IT and development teams are often focused on putting into place production instances of software applications. This is a very different prospect from creating a tool for a single data scientist to use on their own, or even from prototyping a large software application.
This means IT and development teams must build software that is:
….in addition to performing the analysis task or implementing the specific algorithm that you have conceptualized or created.
Getting all these valuable attributes is not free and takes time, so keep that in mind. When looking to hire a software development contractor or company, make sure that they are equipped to handle these critical, but often overlooked aspects of your project. While continuous integration and delivery are not absolutely necessary to meet these goals, it is a great way to ensure that all of these requirements are met in the highest quality manner possible.
Get ready to make great things happen as you work with IT and development to create a software application by following these recommendations and bringing your unique strengths to the project. Taking a team approach, aligning your work with development best practices, and understanding the approach of the team you are partnering with will ensure success in bringing your cool new idea to a wider audience, in a supportable high-quality manner.