If you are starting a software project today there is a seventy percent (70%) chance that it will experience one or a combination of delivery delays, budget overruns, or outright failure. Similar to other types of projects, there are a variety of things that cause software projects to fail in the same way that there are numerous things that have to be managed in order to deliver successfully. To make things even worse, it is very easy to derail a software project and have no idea it is derailing until it is too late.
There are 4 things I’ll like to share that I believe lay a foundation for successful software projects, they are:
1. A clear business goal.
2. A set of requirements.
3. Unbiased estimates.
4. A delivery plan.
These things may seem obvious but they are commonly overlooked and done poorly. Let’s examine each one how together they form a foundation for a successful software project.
Software projects exist to achieve a specific goal or set of goals (even if experimental). A software project goal could be to make sure a piece of software is available by a certain date or to deliver some business value within a given budget. This goal is what will determine the end of the project, and this has to be set and made crystal clear at the beginning of the project. It also has to be clearly communicated to project stakeholders. It is ok if the goal changes during the execution of the project but then, everyone needs to know it has changed and what the new goal is.
It is not uncommon to find ongoing software projects that have no clear definition of what the goal of the project is. It is also common to find a lack of shared understanding of the project goals among project stakeholders.
Not knowing the end target or goal exposes a software project to delivery delays, budget overruns, and possible failure.
To achieve the business goal of a software project, several features have to be built (or not built, just as important) and these features make up the requirements of the project. The nature of the software project will determine if how much upfront work needs to go into the definition of the requirements. Exploratory projects may tend to leave the detailed definition of requirements till later on in the project. Software projects that value predictability (this is of concern to business leaders) would need to have more effort put into the detailed definition of requirements early on in the project.
Real-world software projects will have their requirements change before the project is delivered and provision has to be made to manage this.
Poor estimation leads to delayed projects. Mistaking estimates for delivery commitments also leads to delayed projects. When coming up with estimates they need to be unbiased and molded or forced to align with the business goal. In coming up with unbiased estimates, one has to ignore what the business goal is and simply come up with estimates based on the nature and complexity of the requirements. Doing this and coming up with high-quality estimates that are unbiased will assist with the next step. Planning.
Coming up with a plan is what ties everything together and reveals the true state of what can be achieved on the project. The business goal highlights what the end is, the requirements define what needs to get done, the unbiased estimates reveal how much budget and time is required to implement the requirements.
The delivery plan lets you lay out what can be realistically achieved on the project. Most times you may have to forgo a number of requirements. Businesses value predictability as it is better to know early on what will be ready by deadline than falsely expecting to have every requirement ready.
It is important to put in the time and effort required to come up with a business goal, a set of requirements, unbiased estimates, and a delivery plan. Stakeholders on a project need to work together to develop these. As mentioned earlier, these 4 things lay a foundation and do not guarantee software project success. Software projects evolve as they get implemented and managing all those moving parts is just as important as having a good foundation when starting out. (Video below)
Data centers that are online for public use of shared resources. These resources such as storage, networking, compute etc. are available on-demand.(more…)
As an individual developer working on a personal project, you tend to have a very clear vision and understanding of what you’re working on but the moment another developer joins, everything changes. You are suddenly faced with the challenge of making sure that this new developer understands as much as you do and that they get the whole picture.
Non-trivial projects are built by teams and the challenge of ensuring that everyone on the team understands the vision and goals of the project is what shared understanding is all about. You can also refer to shared understanding as the “why”, “what” and “how” of a project.
In a paper title “Shared Understanding within Military Coalitions: A Definition and Review of Research Challenges” by Paul R. Smart et al. defines shared understanding as:
“Shared understanding is commonly seen as essential to the success of coalition operations. Anecdotal reports suggest that shared understanding enables coalition forces to coordinate their efforts in respect of mission goals, and shortfalls in shared understanding are frequently cited as the reason for poor coalition performance”
Though the definition above talks about military operations it generally relates to any kind of team operation. Software teams also have “coalition operations” (software development activities), “mission goals” (e.g. release version 1.0) and “shortfalls in shared understanding” lead to “poor coalition performance” (in other words, frequent re-work, failed projects).
A team is made up of several people that share a common goal. Failure to buy into this common goal, whatever it may be, leads to failure in the achievement of the goal. This is why having a shared understanding across the team is very important to software teams or any type of team for that matter.
Achieving shared understanding across a team may sound simple and straight forward but in practice it is a lot harder to achieve. Here are some reasons:
Differences in the backgrounds of team members and the level of exposure to the domain for which a problem is being solved significantly influences the shared understanding of the team. Another thing which is closely related to this is the assumptions that people make about certain things when building a mental model of a problem or a solution.
I doubt there is any industry in which the context changes so frequently as with software development. In other engineering practices the context defined at the start relatively remains unchanged throughout the lifetime of a project. A building project to erect a 3-bedroom duplex hardly ever changes into building a 3 storey structure.
In software development though, the context of a project may change as frequently as weekly or even daily. This is the reason why the waterfall approach to software development often fails because unlike other fields of engineering, you may start out building a photo sharing app but pivot and end up building an artificial-intelligence-based-plant-disease-detecting app. The challenge here is that the shared understanding of the team about the context of the project has to evolve at the same rate for everyone including other stakeholders, which is easier said than done.
When working alone, the lone developer is the client, product owner, architect, business/system analyst, UI/UX designer, scrum master, cross-functional developer, tester, technical writer and project sponsor all in one. It can literally take seconds from when a new feature is imagined to when the first UI wireframes are created. It could be just a few minutes before the first lines of code are written and less than a day for the feature to hit production.
As a team a lot of the roles listed above are found in numerous individuals. Whereas it can take a lone developer less than a day to release a single feature, it could take weeks before a team is able to get the first UI wireframes done for the same feature. The main reason this happens is due to the layers of communication (which are unnecessary most times). Typically a team will appoint someone to relate with the client and understand their needs before relaying that information to the team but as with the popular party game of Chinese Whispers (see an interesting video about this here) demonstrates, a lot of vital information gets lost in transit and the understanding the team members get differ from what the client intended. Team members who get the briefing later or are located in a different geographic location get a skewed version of the understanding. In short, the much important shared understanding is not achieved.
With co-located teams, it is a little easier to deal with all the issues listed above. Things become harder when you sprinkle remote team members in to the mix. Achieving shared understanding becomes much harder mainly because remote teams are almost always informed after the fact or never at all and critical bits of information tend to fall off.
At this point you may be wondering how does one achieve shared understanding across a software development team. Here are a few suggestions on how to achieve this:
Try as much as possible to get everyone in for project related activities such as meetings with the client. The reason why this helps in achieving shared understanding is because it flattens communication channels. When the developers hear the what the client needs directly they tend to get a better understanding of what is required and may even see better ways of getting things done. Recording important meetings also helps.
When communicating about how to solve problem within a team, it is important to eliminate vagueness. To do this, the key things that are repeatedly talked about, entities, and domain concepts need to be defined by a single agreed-upon word or phrase. These words/phrases should be made available to all team members (for example, sticking to the agreed words or phrases in all meetings, gently correcting any misunderstandings or misuse, listed on a physical team board).
Meeting regularly helps reduce the adverse effect of constant context morphing. Adopting some agility in your software development approach helps with this. For example, daily stand-ups are great ways to ensure everyone on the team is still on the same page. Regular meetings with the client to demonstrate progress also guarantees that “understandings” are still valid.
When having one-on-one discussions, there is a feedback system a team can imbibe to ensure that there is shared understanding between the individuals involved. Instead of one of the parties involved in the discussion saying “yes, I understand”, the individual should instead explain what they understand and state how this information will affect their work. Doing this simple thing forces the parties involved in the discussion to get on the same page as more often it will be seen that when one says “yes, I understand” it doesn’t always mean that the individual’s understanding is the same as the other’s. This technique can easily be applied to group meetings as well. You can view this as a summary and conclusion section of any team related discussion.
You can speak from here to Mars but until you get some feedback from your audience (an individual or group) you really can’t tell if they understand what you’re trying to communicate. Environments and team cultures that make it hard for members to express themselves will typically have low levels of shared understanding and high levels of re-work. To improve the levels of shared understanding on your team you need to allow team members express themselves freely because this gives you an opportunity to gently correct any misunderstandings. You also need to give extra attention to new team members as they are still learning the team culture and are usually under pressure to fit in that they tend to not reveal their misunderstandings in meetings.
Software development is a communication problem. The key to picking up momentum as a team and maintaining it comes down to how well everyone on the team understands why they are doing what they are doing, and how they plan to get it done. Everyone has to be on the same page and this is what shared understanding is all about. Differences in team member’s domain knowledge, a constantly shifting context, communication channels and geographic location of members are some of the things that make achieving shared understanding harder. Ensuring there are no dumb questions, an instant feedback system is adopted, meetings are held regularly, a shared vocabulary exists and everyone is allowed into the room helps in achieving shared understanding across a team.
Simple reason: You’ll end up with a high-quality app.
Here are some details…
Let’s look at the options you have. People looking to have an app built end up choosing between hiring a freelancer or an agency. At first glance, freelancers generally appear to be cheaper than agencies and most people end up going for this option until somewhere down the line they encounter problems such as the freelancer “ghosting” – suddenly becomes unreachable, it becomes harder and takes longer to make modifications to your app, your app keeps breaking down or certain aspects of it never work as expected, you keep putting money in but get very little value out. All these in addition to the time and effort it cost you to get the right freelancer in the first place.
Going the freelancer route involves a lot of hidden costs and you may have to start your app all over again in a lot of cases. The reason for this is mainly because unrealistic timelines (presented to impress you) force the freelancer to cut corners, sounds and appears ok at first but always, always comes back to haunt the project. It is a very bad thing to do.
There are clear reasons why at first glance agencies appear to cost more to build your app. You need to accept that it pays to invest your time and money into building your app right. Here are some of the reasons why you are better off with an agency.
Building an app requires a lot of specialized skills and that is why the best apps are built by teams. Someone on the team has to play the role of understanding and documenting your requirements and communicating these to the rest of the team. The team needs a software architect who will translate the requirements into evolvable system designs that will guide developers. This is a key role that is often overlooked when dealing with freelancers (and of-course not part of the freelancer’s cost). Without a carefully and skillfully designed system, apps fail to evolve and the very nature of apps is to evolve. Constantly.
Other important skills required on the team are developers who will carry out the actual building of the app. Most freelancers are lone developers with a pinch of other skills. Testers are another set of team members that are crucial to app success and often overlooked. A team responsible for building an app needs to stress-test the apps they build at the code level. The team also has to have quality checks at the requirements and design levels. In addition to the technical skills/roles described, the team also requires some people to support them so that the technical people can focus squarely on delivering their very best.
Keeping standards requires a lot of discipline. Agencies are “going concerns” therefore their relevance and reputation matter a lot, if not the most important thing. In other words, the survival of an agency depends on its relevance and reputation and for an agency to achieve this they need to have well-defined processes for building apps that deliver great results each time. These processes guarantee a high level of output each time, no matter who the individuals who make up the team are. Your needs and requirements go in through one end and a high-quality app, which does exactly what you want pops out the other.
This is one issue that is hardly considered when choosing who should build an app. It is also the most painful one to deal with once your app gets built poorly.
The nature of any piece of software is to evolve. It is easy for anyone to build you the first version of an app but the true challenge is “how easy is it to change, modify, improve, evolve after the first version?”. Once you have an app built, at some point you are going to want to improve it and it’s very frustrating when making changes to your app becomes almost impossible. This is where most people regret hiring a freelancer in the first place.
The solution to this type of problem is to make sure that the app is designed for change right from the beginning. It takes quite some effort & time to ensure an app is built for change, this is why agencies insist on having a design phase for any app that gets built. At Intellectual Apps, we never skip this part due to its importance, or any other part for that matter. The reason freelancers will not bother to insist on the design phase is that they generally think short term. You need an app, they build it, it kind of works, they get paid and that’s it. Once a freelancer gets paid he/she doesn’t expect to be the same person to improve the app in the future and even if you do get a hold of them to do this you’ll notice that it takes ages for anything valuable to be produced. An agency on the other is a “going concern” and knows that they exist to create value now and in the future, so it makes a lot of sense for them and you to invest in the design of the app so it can evolve easily when it has to.
There are some exceptional freelancers out there, in the same way, there are some poor agencies. The fact though is if you need an app built, then you need to think long term and the partner you choose to build your app has to also think long term as well. You need to trade lower costs today for a good investment in your app. You are better off doing this with an agency that has the skills, processes, and experience.
The software development industry has challenges that have plagued it for years. You need to get a partner that acknowledges these challenges, actually understands them for what they are and clearly has processes in place to mitigate them.
Yes, we are an agency. At Intellectual Apps, we help our clients Imagine, Design and Create apps. Get started by telling us about your idea.
Clients that need software built always want to know how much it will cost them, and a lot of times (at least as we have experienced at Intellectual Apps) they want to have this information before work begins. In order to get a “realistic” estimate of how much it will cost to build the piece of software, agencies would elicit the requirements and then based on that come up with an estimate of the cost. Client has a cost for getting their software built while the agency is happy they got the project. Great!
Work starts and in a couple of weeks (in some cases days), the client asks for some features to be included in the project. If a change request management process was put in place before the project started then that process kicks in to evaluate the impact of the change and its priority. Long story short, the client will always find a reason to have more features added and will expand the initial scope by several features. In my experience building software systems over the past 12 years, I have never seen a software development project start and end without the scope expanding. In essence, software development is an exploratory journey, things become clearer only after embarking on this journey. This is simply a fact of software development.
The issue with the scenario described above is that the agency hardly ever gets a corresponding expansion in the cost of the project (which determines their income). Since more features mean more time and effort, the agency simply suffers. One may argue that the change request process be tighter in an attempt to discourage clients from expanding the scope but this rarely works. There is a very high tendency for a client to take the features they are trying to introduce more seriously than all the others put together, refusing to let the scope of the project expand is like taking a sweet from a child, they become upset.
Fixed-cost-custom-software-development projects will always be plagued by expanding scope. Software development agencies generate revenue by offering professional time and effort, so the moment a project takes longer to complete with no corresponding payment for that extension, the agency looses.
From how I see it there are two options:
Option 1 hardly works and leads to all sorts of problems. The waterfall approach is closely associated to this way of building software and the fact remains that it is very hard, near impossible to know and freeze all the requirements for any piece of software upfront.
Option 2 on the other hand is best suited to deal with the uncertainties of software development. It does require the client’s trust in the agency and without trust, it just won’t work. With this option the agency asks the client how much they are willing to “invest” in the software rather than how much it will “cost”. Simply using the word “invest” turns things around as this forms a foundation for the client and the agency to work as partners in exploring a software solution to the client’s problem.
Core to this option is the need for the agency to know their burn rate such that, the agency can easily determine how much it costs them to have a team working on a software solution for a short period of time (say a week, two or even a month). Even though the client is willing to invest x amount of money this approach helps in mitigating the financial risks associated with any software development project by requiring that payments be made for short periods of time to work on parts of the scope of the software. If everything works fine to the satifaction of the client, only then will the next set of requirements get dealt with. The client is free to make adjustments as they wish, as long as they can pay for the time spent by the agency.
Determining the cost of a software development project based on “perceived” requirements is more likely to lead to a project failure. The nature of custom software development is such that a lot of the requirements stated at the start of the project turn out to be incomplete, and once clients get a clearer picture of what they want (after work has started) they expect the project to accommodate such. If the cost of the project was determined based on the initial scope then it becomes very hard to convince the client to pay more.
Based on the fact that custom software development is actually a process of discovery and building, the cost of any project is best based on time. Software development agencies will need to know how much is needed to keep the team running for say a month and let the client invest funds into getting them to work for a defined period of time. Within this period of time, the agency works on whatever the client feels is of the highest priority to them.
Cost estimation techniques should not be confused with software development methodologies. For example a client can pay for 12 weeks worth of development time and the agency applies a waterfall software development methodology in developing the solution. Also note that certain approaches to building software work best with certain techniques for cost estimation. In my opinion, Agile software development methodology works best with cost estimates based on time.
Intellectual Apps’ Founder was recently at the O’Reilly Software Architecture Conference in London, this article is a summary of his presentation.
Maintaining an API gateway these days generally involves handling different types of user facing apps such as a web, mobile (of different platforms) and IOT devices. The Backend for Frontend (BFF) pattern specifically addresses this aspect of software solutions.
At Intellectual Apps we build software solutions, and in creating solutions for our clients we frequently get to build web and mobile apps within a single solution. This often means that we have a single API gateway for all the apps to communicate through. Over numerous clients/projects/products we started to notice a lot of repetition in the modules we built into software solutions, so naturally we sort to find a way we could share functionality across solutions in an easy way and this lead us to start experimenting with microservices.
In building microsevices based solutions, as the name suggests, one ends up having several independent services working together as part of a single solution. This posed a specific challenge for us as we now had to make the user facing apps do a lot more work in composing data from numerous services. So we took a step back and after some brainstorming, we realised that we had several other problems which presented themselves in two forms:
These scenarios created the following specific challenges for us:
Having all apps communicate through a single API gateway.
Given that the data consumption patterns of mobile apps are not the same as web apps, having all the apps share a single API gateway meant that the mobile apps often got more data than necessary when calling the same endpoints being used by the web application. For instance, when returning data to show the details of an item, the web app has room to show way more details than a mobile app can. So when the mobile app calls the same endpoint it simply gets more data that it can possibly display on a small screen which means a lot of the data never gets displayed immediately.
With all apps communicating through a single API gateway, the codebase got bloated. Many attimes this was due to the face that checks were put in place at the API gateway level to determine which app was make the call just so that app specific things could be done.
Having apps communicate with multiple services.
The major challenge with this approach is the extra processing that mobile apps had to do in order compose data coming from multiple independent services in addition to the numerous network calls that have to be made. The immediate effect of this is the drain on battery life.
Enter the BFF
After conducting some research we came across the Backend for Frontend pattern and the motivation for us considering this was to enable us operate in a more agile fashion. The approach was popularized by Sam Newman, the author of Building Microservices. Sam Newman talks about his experience working on a SoundCloud project which faced challenges similar to ours and how he and the team there applied this approach to address the challenges.
We adopted the BFF approach and applied it to our projects and this made our architecture look as follows:
The BFF pattern is a technique where a dedicated layer serves a specific front-end platform. It can easily be seen from the diagram above that each platform gets a specific layer that performs the following functions:
Having BFFs perform the functions listed above means that:
When to use the BFF pattern
If one or more of the following are true in your current architecture then the BFF pattern may be something to consider:
A Little More
Typically the front-end team should maintain the BFFs as these layers are technically extensions of the apps they work on. Doing this will let your front-end team iterate faster as they won’t have to communicate with another team, supposing the BFF is maintained by the back-end team.
It may be argued that there will be duplication of code across BFFs, yes, this holds true but the trade-off here is having bloated code in a single API gateway. We even go to the point of having separate BFFs for Android and iOS platforms as this let us focus specifically on one platform.
It also goes without saying that with the BFF approach, there will be more codebases to maintain.
That’s it and I hope you find this article useful, and consider using this pattern as we do at Intellectual Apps so you could respond faster to client and user requirements.
It’s obvious. Fifteen years ago the saying was that if you don’t have your business online, you’re out of business. Businesses did all they could to make sure that they had presence online by having a company website. It’s obvious that the business climate has changed and today if your business is not connecting with customers via mobile then you are out of business or somewhere close to that.
To better understand this, we’ll continue by answering the following questions:
1. Why does your business need to go mobile?
2. What does going mobile mean for your business?
3. What steps can you take to get your business mobile?
Simple answer: we are now deep into the post-pc era. Longer answer: well, your customers expect it and studies have shown how much time people spend on their devices doing virtually everything. One of the ways businesses ensure that they remain in business is by staying fresh in the minds of their existing customers and by being visible to potential customers. So when people do everything from their mobile devices it makes sense to have your business present in this new virtual world. Another reason why your business needs to go mobile is that you will get to generate and analyse lots of data and in turn use that for more informed business decisions that will ultimately give you an edge over the competition, but most importantly let you offer customers a better experience.
If you sell a product, imagine what this will mean for your business to be able to tell on which days you get the most sales and from which demographic the sales come from, and also being able to know if there are specific things outside of your business that trigger a spike in sales of a particular item so you can prepare for such scenarios. All these are just examples and more meaningful insights can be derived for your specific line of business.
It means positioning your business in such a way that it responds to your customer’s mobile behaviour. The way people consume information on mobile devices is slightly different from how they do it with other sources of information. People are on the go, they want things fast, they have a short attention span and need the right information at the right time. For example, if all your business has online is a website, then that website needs to render correctly and fast on mobile devices.
What your business needs in order to go mobile would depend on the type of business you have and what you want to achieve with your mobile strategy. Some steps you can take are:
1. Ensuring that your current website is responsive. What responsive simply means is the ability of your website to render and display well on different screen sizes.
2. Ensure as part of your mobile strategy to build and release new features with a mobile-first approach.
3. To take things further and in solidifying the interaction between your business and your customers, you can get a mobile app built to meet your specific requirements. If you seek a deeper and richer engagement with your customers then having a mobile app built for your business is the way to go.
4. Another thing you need to consider in going mobile is that it isn’t all about building this or building that, going mobile also has a lot to do with how you disseminate information to your audience. The information you put out there will be more useful to your customers if you consider the context within which they are most likely to consume that piece of information (stay tuned for a post giving more details on this).
5. If you are really serious about this, which we believe every business owner or stakeholder should be, then you can get a competent company to help you come up with a mobile strategy and chart a way to get your business from where it is today to where you need to be. You can start by talking to us.