Personas of a Failing Software Architecture

Personas always interested me. IVR systems have them, Usability folks leverage them, heck even Microsoft had Mort, Elvis and Einstein. I like to think that a good architectural pattern can also be a persona of sorts and the anti-patterns covered by Joseph Yoder in his Big Ball of Mud paper certainly hit home for me.

Today, a business drives the need for software architectures that can confront rapid change. It has become increasingly clear that software architecture methods have evolved in ways that are distinct from the more formal waterfall methods. The pace of evolution in development of systems used to be measured in years, now it is increasingly measured in months. The untapped power of software's ability to meet change head-on is one of its most striking factors, yet it is often overlooked because of the very same business trying to leverage it. It is therefore, an issue that every software designer must address.

Because organizations have different needs, and requirements change over time, it is my opinion that the need to adopt an iterative architecture to build systems that can adapt to change as requirements change is an integral part of software architecture. Iterative architecture methodologies manage change through a series of transitional architectures which aim to deliver a final target architecture in the future. It is much easier to make a course correction having only gone part of the way. 

At any rate, try to laugh a little as you read through these, we have all been there in one way or another!

Shantytown
You need to deliver quality software on time, and within budget; So, you focus first on features and functionality, then on architecture...
Shantytowns fulfill an immediate need for housing by bringing available resources to bear on the problem. Loftier architectural goals are a luxury that has to wait.

Maintaining a shantytown is labor-intensive and devours a lot of development time. Developers are often wasting a good portion of their time to improvising repairs with the materials on-hand: From roof repair to ad-hoc sanitation, its a downward spiral to an unmaintainable system. All too many of our software systems seem to be, architecturally, little more than shantytowns. Administrative tools and infrastructure support are usually inadequate, primitive and infrastructure such as libraries and frameworks, are under utilized or even non-existent. In the wake of this "progress" developers continue to plow ahead, unchecked, to grow other portions of the system without governance. Deadlines hang like vultures and architecture takes a back seat to a hurried fix.

Permanent Prototype
You need an immediate fix for a small problem, or a quick prototype/proof of concept.

Like a temporary storage shed you have every intention of quickly tearing down to replace with something more permanent, prototype code often has a way of enduring indefinitely. The temptation to continue to use the temporary one for "a while" can be difficult to resist.
Often, when you are prototyping a system, you are not usually concerned with how elegant or efficient your code is. You know that you will only use it to prove a concept. Once the prototype is done, the code will be thrown away and written properly. As the time nears to demonstrate the prototype, the temptation to load it with impressive but utterly inefficient realizations of the system’s expected functionality can be hard to resist. Sometimes, this strategy can be a bit too successful. The client, rather than funding the next phase of the project, may slate the prototype itself for release. Do this often enough and you'll have a nice little shantytown over time too.

Sweeping it Under the Rug
Spaghetti code is hard to understand, repair, or extend. If you can’t easily make a mess go away, at least put a pretty face on it. This will help to restrict the disorder, keeps it out of sight, and can open the door to additional refactoring.

If you can’t make a mess go away, at least you can hide it. Urban renewal often begins by painting murals over graffiti so why not do the same to your code? There are valid reasons, other than aesthetics and guilt for trying to clean up messy code. With a looming deadline you've got some code that works, but could stand a bit of architectural integrity. You know there are good ideas buried in there somewhere, but that if you don’t start to make them more evident, they may be lost forever. Eventually, you sweep enough things under the rug and it becomes a monster you no longer wish to see. At first glance, it can sometimes terrify those who would dare try and control it but the first step on the road to architectural integrity is to identify the disparate sections of the system, and consolidate them into logical portions of a bigger system.

Total Rewrite
Your code has declined to the point where there is no point in going on. Repair, or even comprehension is not even a thought; So, toss it and start over.

Sometimes it’s just easier to throw a system away, and start over. Examples abound, but starting over can often be seen as a defeat. I've seen people pulling covert ops the US government would be proud of, to hide project re-do's. One good reason for starting over might be that you feel that you've got the experience you need to do the job properly. One way to have gained this experience is to have participated at some level in the "unsuccessful" development of a previous version of the system. Bear in mind, however, the new system is not designed blindly. It is essential that a thorough post-mortem review be done of the old system, to see what it did well, and why it failed. Bad code can bog down a good design but a good design can isolate and contain bad code. Remember...Refactor!

The Ten Commandments of Egoless Programming

Lets face it, nobody is really smart enough to program computers. To fully understanding most programs requires an almost limitless capacity to absorb details and comprehend them all at the same time. The way you focus your intelligence is more important than how much intelligence you have.

So why do some people place such a premium on trying to purport encyclopedic knowledge on a topic? One word; Ego.

Ego. It's what makes us jump up to answer questions without really understanding the topic. It's what drives us to believe that the other person is just plain ignorant to the facts. I'm right, your wrong and if you can't see things my way... You get the picture. We all know someone like this, but we are all like this in some shape or form. I know I can't help it, but unlike some people I strive to work on it because I'm not perfect.

There is a story about Thomas Edison's early attempts to come up with the right material for a light bulb. He had tried a thousand different elements and all had failed. A colleague asked him if he felt his time had been wasted, since he had discovered nothing. "Hardly," Edison is said to have retorted briskly. "I have discovered a thousand things that don't work."

It is that very same state of mind that is the essence of a great programmer. Every failure needs to be viewed as a learning experience. Every programmer starts out as a lump of coal and strives to be a diamond.. Only one thing makes a diamond; time and pressure.

Some people might be quick to think this "mantra" is part of an Agile mentality. Well, I'm telling you now, your wrong.

The principles of software truly stand the test of time; The Psychology of Computer Programming was written in 1971, the year I was born! In the book Jerry Weinberg established "The Ten Commandments of Egoless Programming"

1. Understand and accept that you will make mistakes. The point is to find them early, before they make it into production. Fortunately, except for the few of us developing rocket guidance software at JPL, mistakes are rarely fatal in our industry, so we can, and should, learn, laugh, and move on.



2. You are not your code. Remember that the entire point of a review is to find problems, and problems will be found. Don't take it personally when one is uncovered.



3. No matter how much "karate" you know, someone else will always know more. Such an individual can teach you some new moves if you ask. Seek and accept input from others, especially when you think it's not needed.



4. Don't rewrite code without consultation. There's a fine line between "fixing code" and "rewriting code." Know the difference, and pursue stylistic changes within the framework of a code review, not as a lone enforcer.



5. Treat people who know less than you with respect, deference, and patience. Nontechnical people who deal with developers on a regular basis almost universally hold the opinion that we are prima-donnas at best and crybabies at worst. Don't reinforce this stereotype with anger and impatience.



6. The only constant in the world is change. Be open to it and accept it with a smile. Look at each change to your requirements, platform, or tool as a new challenge, not as some serious inconvenience to be fought.



7. The only true authority stems from knowledge, not from position. Knowledge engenders authority, and authority engenders respect—so if you want respect in an egoless environment, cultivate knowledge.



8. Fight for what you believe, but gracefully accept defeat. Understand that sometimes your ideas will be overruled. Even if you do turn out to be right, don't take revenge or say, "I told you so" more than a few times at most, and don't make your dearly departed idea a martyr or rallying cry.



9. Don't be "the guy in the room." Don't be the guy coding in the dark office emerging only to buy cola. The guy in the room is out of touch, out of sight, and out of control and has no place in an open, collaborative environment.



10. Critique code instead of people—be kind to the coder, not to the code.As much as possible, make all of your comments positive and oriented to improving the code. Relate comments to local standards, program specs, increased performance, etc.

Top 10 XML Specifications Rejected by the W3C

Got this in an email. for those of you who work in XML, its funny.

10. WS-IrishSpring: for scented, more pleasing SOAP packets

9. WS-UPS: for sending SOAP packets in real envelopes

8. WS-USPS: for sending SOAP packets that dont need to get there

7. WS-PrisonShower: for picking up the dropped SOAP packets

6. X-Wife: protocol for monetary transfer

5. WS-Insecurity: dating protocol for web services programmers

4. WS-Monopoly: protocol used to keep antitrust penalties to manageable levels

3. NICKLE: for encoding smaller binary attachments

2. SFFCI: Syndication Format for Complete Idiots

1. WS-XXX: bringing a business model to XML.

Domain Driven Design

With the advent of agile development methodologies, many of their edicts have been filtering back to the design stages in the form of what is now called "Domain Driven Design" (DDD). Concepts like group brainstorming design sessions, on-site customers and iterative development have had a dramatic affect on the realm of software architecture.

The days of the "waterfall" mentality are dwindling, people are looking for a fresh new approach to software development and with just reason. Business is an organic entity, ever changing and evolving to meet the needs of its market but the software it uses is often limited because it only meets the business' current needs. The process by which it was developed is flawed, people spout promises of extensibility but I'm not sure we are all on the same page when it comes to the definition of that word.

With waterfall, the Subject Matter Experts (SME's) talk to the analysts, who, in turn, create a multitude of charts and diagrams representing the business process in a more abstracted "technical" language for the developers. The problem with this model is the singular direction in which information travels. The SME's and the analyst learn nothing from the development staff. As well, the information that filters from the SME's to the development staff minimizes their exposure to and understanding of the business. When your development staff does not understand the business, there effectiveness is altered and their tasks become features instead of principles.

Agile is not immune to this dilemma. Development teams get so caught up in the feature list for a given iteration, they become nothing more than an feature assembly line, only pausing long enough to ask, "What's next?"

Feature, rather than principle driven software, can be robust; particularly if the development team practices refactoring, but it will rarely benefit from a system developed with the understanding of the business. Subtle decisions made in the design process can be the difference between a system of rich domain objects and anemic ones

Developing a robust, domain driven model is a bit of an art, but the practical design and implementation is relatively systematic. By isolating the domain from the mass of concerns in the implementation of the system will greatly enhance the connection between the domain and the software derived from it. Definition of model elements from the domain will sharpen their meaning and provides for a more practical design implementation.

It's Like Riding a Bicycle?

I have been struggling these past few days to "define" a concept I have been doing for the past 4 years, a "Domain Pattern". It really is more that a software pattern, its an entire process pattern for designing and developing software.

I never realized how difficult it can be to try and impart "inate" knowledge to others; It is turning out to be a fairly difficult task. It is one I liken to trying to explain to someone how to ride a 2 wheel bicycle. How do you explain balance?
True, you can convey certain concepts with words, but the actual development of balance must be learned through trial and error.

I think I need a mental rosetta stone...

Pay Phones and Pointers

I recently read an interesting little article on the upcoming college class of 2010. It expounded on how, among other items, this upcoming generation of college grads never really knew what it was like to search for a pay phone. Imagine that! I did and it made me feel old, so I shifted my thought to a tangent. Imagine the latest generation of developers, working in managed code environments who don't worry about pointers, malloc or deal with heap corruptions.

Managed code has done a lot to alleviate the pains of development. We produce code faster (not faster code) and without the worries because the managed code runtime does it all for us, right????

Well, we can always add more memory......

SOA Business Processes: Strive to Achieve (Part 1)

We rolled out the first version of our SOA about a year ago. It was a big effort and a huge change in direction for us as a company. At that point, it was a brave step as SOA was more of a concept than a reality in most people’s eyes. But we did it. V1 of the SOA was just what you might expect, total vertical transparency for a few core systems and data stores. Nothing fancy, just a collection of services exposing consolidated logic. By itself, for us, it was a huge step in the right direction. We now have a single code base with which we can interact consistently with core systems and the interoperability of web services so we can extend this functionality to other platform and vendors.

Sounds great right? OK it is, but it also seems to have its own drawback. You see, now that this SOA, admittedly still in its infancy, is seen as a viable solution, the drive to push applications through it is strong. This is great, save the issue of business intelligence.

There are two sides to the SOA development coin as I see them; Service Orientation (SO) and Business Intelligence (BI). SO represents the vertically transparent services which expose the functionality of a system or individual stores of data within the enterprise as services. There is no defined interactivity between these services, the messages are simple and small, and the service interfaces are very coarse in their granularity: AKA Information Silos.

Business intelligence, however, is the aggregation of these granular services spanning multiple systems. A layer, if you will, that spans these information silos and they are driven by the processes defined by the business. To me, this is the biggest payoff for an SOA concept and where you really start seeing the opportunities to ‘trim the fat’ from the existing processes. Lets say you have a process for customer address change, where there are several key systems maintaining independent profile data on a customer. When it comes time for that customer to change his/her address, it must be replicated throughout many systems, across many departments, by many people. Even better, it needs to be done accurately; Remember, To Err is Human! Add any amount of logic necessary for sending out notification letters or flagging records and you’ve got a process worthy of SOA implementation in the form of BI.

The payoff is simple, once you have this process coded for and exposed; you’ll have a consistent automated process for changing address. The process can then be offered publicly without any worry as to process adherence by the subscribers. It’s all there. Of course there are lots of contingencies to deal with in the real world. For instance, processes change right? That’s where the choice in architecture to orchestrate these processes comes into play. Orchestration is a topic unto itself, one I will be covering in a different entry.

Other peoples code can be fun too!!

A while back, a colleague got me hooked on this site. The entire purpose of this site is for fellow programmers to share some of the worst code they have found. Most of the code is "legacy code" left by previous programmers. If you have ever run across code and said "WTF!?!?!" then it deserves to be on this site.

WinFX Here we come...

One of the best parts about where I work is our ability to quickly adopt the latest technology and apply it to out infrastructure. Hell, we started developing the second version of our SOA using the 2.0 framework while it was still in beta!! Now, this does not mean we are pushing beta software into a production environment, that would be foolish, but it does seem to make us early adopters of technology. And it sure does make programming interesting again.

So here comes WinFX, which finally has a Go-Live license. I have been waiting forever for this product to get close enough for me to consider using. Specifically, I have been waiting for Windows Workflow Foundation (WWF) which really fits the bill for a lot of complex functionality we have been planning for our SOA.

If you have never played with WWF, I suggest you DL the components and try it out.

"Windows Workflow Foundation is the programming model, engine and tools for quickly building workflow enabled applications on Windows. It consists of a WinFX namespace, an in-process workflow engine, and designers for Visual Studio 2005. Windows Workflow Foundation is available (currently in beta) for both client and server versions of Windows. Windows Workflow Foundation includes support for both system workflow and human workflow across a wide range of scenarios including: workflow within line of business applications, user interface page-flow, document-centric workflow, human workflow, composite workflow for service oriented applications, business rule driven workflow and workflow for systems management.
The Windows Workflow Foundation namespace in WinFX is called System.Workflow. Windows Workflow Foundation provides a consistent and familiar development experience with other WinFX technologies such as ASP.NET, Windows Communication Foundation and Windows Presentation Foundation. Windows Workflow Foundation provides full support for Visual Basic .NET and C#, debugging, a graphical workflow designer and the ability to develop your workflow completely in code. Windows Workflow Foundation also provides an extensible model and designer to build custom activities which encapsulate workflow functionality for end-users or for re-use across multiple projects. Windows Workflow Foundation will be used across many future Microsoft products including The Microsoft Office System, BizTalk Server and the Microsoft Dynamics Products (previously known as Microsoft Business Solutions products). Most applications can benefit from the asynchronous state management features of the workflow model, the rapid development features of the designer, the potential for end-user flexibility, and the increased visibility into run-time code execution.
A workflow is a set of activities stored as a model that describe a real world process. Work passes through the model from start to finish and activities might be executed by people or by system functions. Workflow provides a way of describing the order of execution and dependent relationships between pieces of short or long running work. While it is possible to write a workflow completely in code, workflow is often best viewed graphically. Once a workflow model is compiled it can be executed inside any Windows process including console apps, forms- based apps, Windows Services, ASP.NET web sites and web services."

If you understand the above Microsoft quote, you can understand my excitement!! Not only will this boost out development time in system aggregation, it could play a huge role in giving the clients more flexibility in defining their own custom workflows and submitting them, rather than having to solely rely on the workflows available. Less SOA deployment, development...More Flexibility, Scalability etc.. I'll be posting more experiences as I have them.
See, I'm excited!!

GMail gets a DELETE!!!

It's amazing how such a simple item could garner so much excitement, but I sure did miss having a Delete button.