When planning your serverless project, especially in the early stages of a company diving into the serverless technology world, the goal of any IT team is to build a system, show that they exceeded expectations and build confidence internally in IT as well as from business partners. Having success in these three areas can lead towards a sped up transition to serverless technologies, increased capacity as you eliminate efforts towards system maintenance, and an increased appetite by the business team to take on bigger challenges. That's the best-case scenario, and it is achievable. What happens more often is the project extends four to five times its initial estimates, the team loses the confidence of the business and ultimately gets cut or loses the support it needs to succeed. All of this happens because of five avoidable but common mistakes teams make when planning and executing projects with serverless technology.
It's exciting to see a team get introduced to serverless technology. When they grasp how it simplifies their work, their thought process and more, the team gets excited at the prospect of applying this technology to their current challenges. This is a great and exciting place to be. So how do we go from this to where your business team has lost faith, your team's confidence is shaken, and you 're about ready to pull the plug on a project that has literally cost you hundreds of thousands of dollars? The answer to this is that your team made one or multiple of the following mistakes.
The project is too vague
Too much scope
Underestimating skill sets not being fully developed
Old habits are hard to break
The project is an exception to standard procedures
These mistakes happen so often that they're not tied just to serverless development and often have the effect that the business team will be more comfortable purchasing rather than letting IT build systems. The good news is that these mistakes are avoidable, and there are techniques which can help you and your team be more successful.
Mistake #1: The project is too vague
The number one mistake I've seen are projects which don't have a definition for what success looks like. I've seen projects kicked off where the business unit was only aware of the topic the IT team was going to be working on. It's not uncommon to have a project kicked off without knowing which business units need to be involved, and who on each of those teams should facilitate getting answers. On the IT side, I've seen too many projects where only IT could comprehend the business value that was going to be delivered. It isn't until the middle of all these projects is where things go south. It happens when the business first comprehends what the IT team has constructed and how far off that team is from the reality. From here there are two potential results. First, the business gets more involved, and the initial project estimate is shot. The second potential result is that the business rejects the effort and looks for a solution to purchase. In this second case, the new system usually costs more, requires just as much effort, but since the business is committed and has budgeted for it, this new project is successful.
Solving this issue
So how do you ensure projects aren't too vague? Before project kick offs, it's important to have a stated goal, a good understanding of current business process, the desired result of the project and the general cloud architecture thought through. While this might sound daunting, it's not as bad as it sounds.
The stated goal
For the stated goal, we draw a bit on Enterprise Architecture concepts of "Architectural Vision". This goal is a brief description of what is trying to be accomplished and can be anywhere from a sentence to a paragraph long. Its purpose is to describe the goals of the project and intended outcome at a high level and get buy-in before applying significant effort. Business units will aid in its construction and review, and all stakeholders should sign off on it. This starts the process of clarity, as you would have had to find all the stakeholders and get them all to agree on a purpose. This also helps focus the later items on our list.
A good understanding of business process
The second step to this process is understanding what you have, both from a system perspective and a business process perspective. What is it that is working for the business? Who are the individuals involved in this process you're building a solution for? What are all the touch points where one department or person hands off their part of the process to the next? This part helps provide the team with clarity around some details of how the business operates. If documented well, it can also help point execution team members at who they should talk to for different parts of the process.
The desired result of the project
This is harder than the previous two. One of the major contributors to a project getting done on time is defining a reasonable set of process improvement goals. This could be something as simple as, "We need to view financials and make adjustment to distributors' commissions based on financial or contractual obligations." Getting a clear understanding of what outcome they would consider a win will help the team from going down the rabbit hole of:
What if business wants us to improve their processes?
What about doing integrations to other financial systems?
Etc.
There are thousands of things that you could do in serverless, but the goal of your project needs to stay small, and while it's always good to improve the process if possible the project execution needs to push those off to iterative improvements after releasing the new serverless system.
Building a general cloud architecture
After understanding what success looks like, the last piece before you kick off a team is understanding how to build your project. Good solutions are built to reduce the amount of custom code your team will have to write and reducing the cost of the project by using off-the-shelf services and products. It will also identify what tools and skill sets they need in the team's staffing and training decision making. This is also the stage where project managers, product owners and scrum masters can get a good idea about integration points that could be a risk to project timelines. An example of this is where another team needs to do work for the project, but is already over committed to other projects for the coming months.
Mistake #2: Too much scope
The second big mistake that is made in approaching projects is the team taking on too much scope. Scope can take the form of features, training, supporting systems like builds, time taken investigating issues, documentation and various other activities all of which are important. What this looks like in a project with too much scope is that the team starts off optimistic that they can execute on the project in a general timeframe with confidence. As the project goes along, there are random delays that block forward progress on the team. These random delays also are contingent on a single discipline on the team. The delays can last anywhere from a week to months, continually pushing out the project timeline.
Solving the issue
The solution in this space starts with leadership, but also involves the diligence of everyone on the team. The goal of the team and leadership is to reduce the amount of scope as much as possible for your project. Successfully doing this will focus more of the team's time on project success, and much less on all the other things that just have to be done.
Plan for training
The first step here is to use the architecture developed to help focus the team's education on the specific services and tools they'll need to complete the project. This provides training that the team can leverage immediately, rather than broad training that the team will forget due to lack of application. Internal teams which do technology training can put together customized training plans, but vendors like Trek10 and A Cloud Guru have significant experience training teams with hands-on labs which tie directly into their projects your teams are going to take on.
Eliminate effort around common problems
While eliminating excess training can be a simple decision, other aspects can be tougher, such as a build and release system and simplifying how long it takes to investigate issues.
The following is a list of capabilities and resources most teams have to have in place to be successful.
Getting a head start on defining the services or tools the team will use for topics on this list early in the project can save a ton of headaches and delays later in the project. This is especially true if you can find vendors that reduce or eliminate sections of this list. Leveraging other internal teams can help spread the load, such as build teams and tool teams. Vendors can be critical though in helping reduce the number of these the team has to take on. An example is using the combination of Stackery and Epsagon can help eliminate over half this list for your team. There are other plenty of vendors out there that can also fit in to help eliminate other areas as well. The goal is to remove as much of these as possible from what the team has to build or learn for themselves, and have them spend their time on completing critical pieces of your project.
Continually evaluate to ensure team is focused
Finally, there's the problem of the team taking on too many features. The tendency of development teams is to replace repetitive manual efforts, even to the point of excess. This is healthy for a development team as this leads towards innovation and improved business processes but has the tendency to extend project timelines. To solve this issue, it's important to reenforce the difference between a project for a new system and iterative improvements. The goal of a new system is to get necessary components built with minor innovations while setting up a foundation for future innovation. Using the individuals or documents which can describe the business process along with the list of outcomes desired allows the team to course-correct and save significant innovations for future releases.
Mistake #3: Underestimating skill sets not being fully developed
Any team that starts a serverless journey is going to have a lot of learning to go through. The problem is at the same time it's really easy to grasp how quickly building systems and features will be once they have experience. This leads to over-optimistic estimates on the size and duration of new projects.
Here are some examples of what this can look like in practice:
The team runs into more and more complex "hard" problems which are very technical. This causes delays that can last anywhere from a day to multiple weeks.
In planning, the team takes on "innovative improvements" that they say are easy, but a couple of days turns into a couple of weeks
The team keeps changing their approach and refactoring work already completed, taking anywhere from one to two weeks at a time to complete.
The team is spending a lot of time trying to plan for concurrency, which is taking anywhere from days to weeks researching.
There can be various other indicators, all of which point to a natural evolution of your development team experience. The problem happens when it starts significantly affecting project timelines.
Solving the issue
First, these are normal problems for teams embracing new technologies and methodologies. The goal here is to understand and potentially reduce the time it takes for the team to reach competency in serverless. There are two solutions to this problem: hire mentors or plan for delays.
Hire mentors
The first solution is to hire a vendor which has extensive experience and can continually mentor your team and get your organization's serverless experience jumpstarted. The reason this is the first solution is that it's the most cost effective and yields the best results.
Lets take this example:
You have a team of 3 developers and 2 testers. The average burn rate of this team is $49K when taking into account taxes, bennefits, equiptment, etc. You have a project that is expected to take 3 months.
Without a mentor:
Since your team doesn't have mentors in place, they spend at least double the time to complete the project.
Team: $49 x 3 months x 2 learning while building = $294K
Total: $249K+
With a mentor:
In this case you're paying extra for a mentor to be part of the team. You negotiated the vendor to provide you an expert with years of experience building systems like yours and you asked for quotes for 20 hours a week and 40 hours a week
Team: $49 x 3 = $149K
Vendor (Low side): $350 per hour x 20 hours per week x 4 weeks per month x 3 months = $84K
Vendor (High side): $300 per hour x 40 hours per week x 4 weeks per month x 3 months = $144K
Total: $233K - $293K
Result:
What this shows is that even paying for top dollar support for your team, you're actually paying less than you would without that support. You also get the bennefit of having your system created with an eye on future iterative feature development, cutting down the cost of that future work as well since your team won't also be working on refactoring at the same time. While your company has a lot of really smart people, just like any other technology it can take years to learn and and master serverless. Having mentors who help review plans by your team and offer advice helps build good habits from the beginning, and avoids early mistakes. It also provides your team with a sounding board for their own innovative ideas, which a good vendor will encourage and nurture.
Plan for delays
The second option is to build in time for this growth internally. This option by far is the most expensive option, but by building in the expectation that a project will take two to three times as long, you avoid the stress and perceived failure of your project. In order to do this successfully, your team will have to have the discipline of accurately evaluating when stopping to rework is preferable to incremental change. The result of your project will vary in this approach as well. Teams that go this way will regularly have to rework their architectures based on new discoveries in later feature releases. The positive benefit of this approach is that your team will gain experience running into some of the more interesting aspects of serverless. In a company I worked for we joked about having badges for problems we personally ran into of which we had previously heard about, such as eventual consistency, lambda storms, and others.
Mistake #4: Old habits are hard to break
This mistake happens when a team hasn't mentally transitioned to move into serverless approaches and technologies. Serverless breaks away from more traditional approaches, such as service-oriented architectures, three tiered architectures and relational data storage. These more traditional approaches have been engrained in your most experienced developers, but only loose translate in serverless.
Here are some examples of what this can look like:
Your team is building everything in classes in shared libraries, even if a single function only used the capabilities
The team designs a multi-table non-relational database with all fields at the root of the objects, then must query each table to pull out data in order to return a single result.
The team uses a relational database which is only used for storage and retrieval
The team tries to use a single service for problems that don't match, rather than using a variety of services that match the problems they're trying to solve
The team spends time building technical solutions for concurrency issues rather than leveraging services to solve the problem
The team and leadership reduce the number of functions written to simplify the architecture while the amount of code written increases by 3 times.
These show that the team is struggling to understand significant concepts in serverless solutions. The result is the timeline extends for your project as the team is putting more time into the initial development, refactoring later in the project, or refactoring in future feature development.
Solving the issue
The solution to this problem is having a good solution architecture up front to help clarify what the team needs to produce and getting the right training and mentorship.
Solution Architecture up front
Getting a solution architecture up front will help the team understand how and where data is stored as well as what type of programming is nessisary to accomplish the project. It can also help the team understand how to translate their previous experience into the new concepts. This puts more emphasis on having someone who has significant experience with serverless setup your architecture. They will also then become part of your education and mentorship support for your team.
An example of this is in the diagram, where a web browser leverages AppSync to directly store data in a single DynamoDB table. In this example the DynamoDB table stores information about a person and their contacts, with no second table for the list of contacts. This results in the team focusing all of their efforts into the definition of what they are trying to store and in the code in the web browser.
The right mentorship and training
Getting the right mentor for your team can work wonders. A great mentor will help review and refine your team's approach to serverless solutions, and help get your team past misperceptions quickly with minimal impact on the project timeline. This type of mentorship differs from training, as the individual providing the mentorship will need to stay up to speed with goal changes and approaches to technical issues in order to provide the best guidance. The cost for good mentorship will reduce the overall cost for projects significantly, as the mentor will keep the team away for costly mistakes and rework. Vendors like Trek10 can provide broad support with experts in various designs and technologies, giving your team an expert who they can reach out to.
Mistake #5: The project is an exception to standard procedures
The last big mistake when tackling a serverless project is believing that it differs from any other project. It's easy to plan for a small project, pull together a small team, and assume that normal planning and estimating processes would hinder delivery. The result of this is that leadership estimated the project for two months and as the team gets closer to that delivery date they're not even finished with the basic capabilities. What's worse is that no one really knows when the project will finish. With no real understanding of project completion, the team continues to struggle until leadership assigns more and more resources to the team or even cancels the project entirely.
Solving the issue
Whether the team practices scrum, kanban or waterfall, it's important to go through the regular steps associated with these processes. With scrum and kanban reviewing work and estimations in story points help with projecting delivery dates and adjustments as work gets added or removed. It's also important early in the project with these two methodologies to understand the conversion rate from research story points to workable stories and their story points. This ratio tends to be reliably consistent for a project and the team can discover it early on as there's typically more research early in the project. Knowing this ratio will help the projections include currently undefined efforts waiting on research to complete.
With waterfall getting work breakdown thought through up front helps define project timeline planning, with buffer added at the end for unexpected delays and efforts. While waterfall puts more effort up front, in the first phase, most of the research has completed allowing fewer expected tasks. When tasks start taking longer, waterfall starts being able to predict timelines based on the ratio of estimated work to actual work.
In these methodologies, there are checkpoints which will help identify when a project is expanding, provide projections for completion and allow leadership to intervene early on to adjust by reducing scope, increasing team size or adjusting expectations.
Conclusion
While there can be various other influences on a team's delivery, these five are the most significant to organization's attempts of implementing serverless technologies. Using approaches that minimize these helps your projects and organization get setup for success while also driving down the costs and efforts. Leverage vendors to help you build your team's experience, and help eliminate early bad habits and mistakes. Finally, don't fall into the trap of thinking since it's a new type of technology, it shouldn't fall into your standard project planning practices.