Continuous Integration in Agile Software Development

Question:

I am a fresher and have recently joined an IT firm as a .NET developer. After 15 days of orientation and training I am not part of a project team. I have been hearing a lot about Continuous Integration. Can you help me understand what it is and how does it help the project execution?

By: Rajesh Narayanan

Answer:

One of the important reason behind the software project failure was that the defects or issues discovered in the later phases proving to be too expensive to address and thus adding to the time and cost of the project. The industry has come up with different methodologies to address such issues and thus Agile Methodology has gained much adoption and recognition. As you may be aware, one of the key principles of Agile is to put in processes and techniques, so that the defects or issues are detected early on and thus giving opportunity to address sooner. Practices like SCRUM, Pair Programming, Test Driven Development are all expected to address this issue of catching the defects early on. On the same lines Continuous Integration and Deployment also contributes towards offering an opportunity to discover issues early on by offering the customer a working software product every now and then.

Traditionally, developers work on building components or modules independently and there will be a phase which will be fairly closer to UAT for integrating these modules together. No wonder, the development teams will discover many inconsistencies leading to the modules not working in tandem as intended. This calls for rework on one or more of the already built modules.

Continuous Integration addresses this problem by ensuring that every time the developer checks-in the code changes, the code modules are integrated with other related modules and deployed as well. This activity is best achieved by automating the build and deployment process. That means, the developer as he checks in code changes, will see the build and deployment errors and or warnings, which are expected to be resolved then and there, so that every code change that is checkedin does not break the working product. Thus the end customer will be able to see a working product always and able to validate the change as it is implemented.

The following are some of the key practices that makes the Continuous Integration workable and useful as well:

• Maintain a Single Code Repository - Continuous Integration won't work if developers maintain isolated code repositories for different component or modules. There are plenty of open source and commercial source code control tools that offers robust conflict resolution techniques and facilitates usage of single code repository.
• Build Automation - The Continuous build and deployment activity is best accomplished by automating the same. Most of the programming languages support this requirement and tools like CruiseControl are also very useful to accomplish this.
• Test Automation - The automation process shall also execute the basic tests by itself. this where Test Drivent Development is in use. The developers, as they write code are also expected to create test scripts, so that these will also be executed during the automated build process.
• Frequent Code Commits - The developers should commit the code more frequently and at the least every day. They should commit the code after every logical completion of a code change, so that the build does not break. This where, the Agile principles revolves around braking the coding tasks into smaller units, so that such tasks typically take 8 or 16 hours to complete.
• Automated Deployment - Coupled with build automation, the code should also be deployed on to the test environment, so those involved in testing see the latest changes as soon as they are committed into the repository.

Hope the above explains the purpose and the principles of Continuous Integration. You can look up Martin Fowler's article on the subject to know more.

How to Integrate a POS device in a .NET Windows Forms Application

Question:

One of our recent project requires integrating our Windows Forms application with point of sale devices like barcode scanner, check reader etc. I could find out that there is a POS.NET library that can be used for the purpose, but it seems to have been built for .NET 1.12 and no updates after that. Can you suggest me if there is any other .NET libraries that can be used for the purpose. You may also help me by sharing your thoughts and ideas.

Question by Selvamurugan N.

Answer:

Answering your first part of the question - Why there are n't any updates to this library? - I can only guess the answer for this and that I feel is due to the fact the device manufacturers lag behind in updating the communication protocols, which is more so because typically these devices are built for a low cost specific purpose operating environments. For instance, the POS terminals don't have to be on a Windows 7, it would just be enough for them to be on Windows XP or even older operating system. That probably is the reason, why the Library continues to be based on COM components and does not need an update.

On using the POS.NET library, yes, you can still use it in the later frameworks of .NET, but be aware of some of the constraints or issues as described in this blog post. The blog author calls out issues with 64 bit environment and also with .NET Framework 4.0. Don't miss out the comments, as there are some valid opinions and observations in the form of comments. You may also want to check out the Microsoft's publication on using the POS.NET titled as Creating a Proof of Concept POS Application.

If that does not suit you, then go ahead and learn to access and work with USB devices using HID (Human Interface Device), but you need to know the communication protocols and the data structures as specified by the device manufacturer. This will for sure will give you a better control over your integration, but you would be re-inventing the wheel. Here is a nice introduction to working with USB devices in .NET.

My recommendation is that check out if the device vendor provides you a .NET or a COM Library and if yes, using that will be the best option as some device manufacturers may have built certain proprietary protocols, which would work best with their own drivers. If that is not the case, then evaluate POS.NET whether it fits your use cases. Most of the times it should be fine, unless you meet up with some of the constraints when using with later versions of .NET Framework. If left with no choice, use HID interface to have your requirements met. You may also want to check out this msdn article on using Alternate Input Devices.

Enterprise Architecture - Conformance Vs Performance

Question by Ganesh Ramamurthy

I was hearing Erik Dornenburg of Thought Works in a on-demand InfoQ presentation on "Questions for Enterprise Architect" in which he was discussing about Evolutionary Architecture and was calling out Conformance as one of the issues. We have as many frameworks and standards in place and naturally we always aim to conform to these standards, practices and frameworks and even comply with legal framework. He was considering the case of overseeing the developers to conform to standards as parenting the kids. While it is agreeable that with the fast changing business and technology environment, expecting conformance may be a barrier to innovation, how can we ensure that we produce high quality products. Taking cue from this, I was trying to see if Performance is the answer to the Conformance issue. Your thoughts on this subject is welcome.

The Response

Ganesh, thanks for bringing this topic for discussion and yes, that was a good presentation and is a must watch for all aspiring Enterprise Architects.

Erik in this presentation suggests not to attempt building and enforcing standards and frameworks first and recommends that let these evolve over time in line with the business needs. His comparison of the Enterprise Architect to the City Planner or even latter a Gardener as against the a building architect was very apt.

Getting to your point, he even discusses a case of having the code unit tested by the developers and he highlights how conformance may not solve the problem. With conformance, the teams may just work towards just meeting it and no further. In the same example as he says, if 100% code coverage is required to pass the code move to the next phase, then yes, we may get the results of 100% coverage, but still the unit tests may be incomplete or ineffective in other contexts, like logic tests.

In a broader sense, conformance may be necessary in a highly mechanized or high productivity organization, where it is just the efficiency that matters. But this gives little or no room for doing things effectively and even differently fostering innovation. This is where, depending on the level at which the team operates, the standards or rules should be as open as possible and there must be a feedback loop to review and revisit the standards and rules constantly. Like we have moved from a paradigm of hard coded programming style in the early days of software programming to highly configurable software products, the standards and rules should evolve to a state where they are flexible in adoption for the changing needs.

As Erik also says in his presentation, Architects should refrain from detailing the micro level aspects of the systems and should leave it to the developers. This will go well based on the trust that the architects have on the development team, and that is where the Enterprise Architect has to consider or has a role to play in the Organizational culture and in building the necessary trust with various teams and if this important aspect of trust is missing, then conformance is the one that works better, but not any better than just meeting what was required.

In practice, we see developers with little or no accountability produce inferior code and this is what prompts us to put checks and controls all around and thereby not allowing the developers look beyond what is told to them to do. While just allowing them to do things in their own way is the other extreme, there should be high level norms or guidelines and allow the teams to leverage options without violating the higher level norms. This does not mean that such high level standards or norms should be rigid, and this is where the Enterprise Architect should be receptive to the feedback from all channels and should be more than willing to change these norms as we move on.

It may still be necessary to manage teams at the micro level, but the better way to look at this is about grooming or hand holding the teams to get upto the expected level of performance and then leaving them in a space where they can perform rather than just conforming.

Another way to look at this issue is the way we look at the popular saying "Operation Successful, but patient dead". Wherein conformance expects successful process performance, but it need not produce the desired outcome. So, it is important to measure and focus on the outcome as against the focusing on the conformance to the process.

While the functional or business requirements are written down and signed off, it is the non functional quality attributes of the software products like scalability, performance, maintainability etc where the developers tend to follow diverse practices, resulting in compromise in some such abilities of the systems in the longer term. The Enterprise Architect and in turn the Technical Architect or such other intermediate roles should appropriately interpret and translate the enterprise business strategies into the priorities for these non functional attributes and then let the teams to leverage other artifacts as they deem fit but at the same time not to ignore these priorities.


Yet another related issue which will help address the conformance issue is to have a well defined roles and responsibilities and when authority is delegated to roles, it should be coupled with the constraints attached to such delegation. This will help people perform their roles within the given constraints, leveraging the options they have with them to produce the expected outcomes.


This is a broad topic and applies not only to IT or the development function and this is now increasingly being looked at the enterprise level and even the Boards are expected to perform and not just conform. Change is at the center of everything that we have been talking about and an organization where every member realizes that change is the one thing that is constant we can see Performance as against just Conformance.