Continuous Integration

The time between a defect being entered and being found is related to the expense of fixing it

Graphing the number of tests over time would seem to be a good idea

Sligo dashboard: classes, LOC, duplication, max complexity, tests run, line coverage, branch coverage, FindBugs violations; PMD Violations; Max Afferent; Max Efferent

Code reviews are good for high level details; use machines to do the low level detail finding

Code complexity tools: CCMEtrics, Vil

Code duplication detector: Simian

Dependency: NDepend

Coding Standards: FXCop

How to work with an open source team

"Free as in Freedom"
"Free as in Beer"

http://opensource.org/

Q: Don't people resent when companies take open source projects and make money off of them? A: More power to them! Many companies are using their employees to do open source. It's good PR for a company to have people who work on open source - give back to the community, attract high-power talent

Don't contribute unless you:
Know the project license
Get permission from your employer
Get legal review if needed
Can communicate clearly in the project language (usually English)

Oracle tried to strongarm Linux, got squashed, came back with offers to help. Good PR! (I'm not familiar with this story!)

Project currency is trust and respect. You don't start with any. Remember, if you're good, you don't have to point it out.

Q: How do you start gaining respect? A: Post to the mailing list, point out bugs AND fixes. Maybe someone will request a patch, provide it

Q: How does the code stay consistent and looking good? A: There are tools, or people who do work to make things consistent. Or, it doesn't :)

Q: How do you get non-coding contributions going (docs, images)? How do non-coders get cred? A: Projects should support people like tech writers, if they're good.

How to Gather Customer Feedback

Don't aggravate customers with annoying surveys!
Make sure to ask "Is there anything else?"

Several stories about bad feedback forms

Interviewing 5-10 customers is probably as good as interviewing hundreds

Don't assume that no complaints = customer satisfaction. They may just be putting up with it, especially if they feel no one is listening.

Don't just do surveys. Use different feedback-gathering methods. Invite open-ended feedback, in surveys or otherwise.

Don't ignore the feedback!

Focus on the service attributes most important to your clients. Don't know what's important to them? Better find out!

"What aspect of our service is most important to you? Regarding it, how are we doing?"

Lots and lots of examples of how not to

FBWA (Feedback By Walking Around)

I love how Microsoft gets all Ajaxy with feedback on every page: http://msdn2.microsoft.com/en-us/library/ms229931.aspx

Again, act on the feedback! Summary of responses, detailed responses, action

Don't forget power of the naked eye. Often problems are obvious and don't need surveys

Security code reviews

Foundstone Security Frame
Hacme Casino http://www.foundstone.com/us/resources/whitepapers/hacmecasino_userguide.pdf
Foundstone CodeScout

Paros (web app security assessment) http://www.parosproxy.org/index.shtml

Don't overanalyze. (Spending two hours determining if a strcpy is vulnerable. Takes two minutes to change)

Identify code review objectives (Insider backdoors, compliance with specific regulations)

Lots of discussion of tools. I think the point is, use available analysis tools before bothering with a code review - it's easier and cheaper

http://www.securecoding.org/list

http://codesecurely.org

Usability by Inspection

Doing code reviews? That's good! Code reviews are a big help. They ensure uniformity of code, teach people new design patterns, and often even help to avoid bugs.

Doing usability reviews?

Me either. But if you've got a product that has a UI, an easy thing to do to improve the product is to just sit down with a few people, maybe some that will actually use the product, maybe some managers, or maybe just some people that you can pull in to see what they think. That's more or less the gist of what I got from Larry Constantine's session on Usability Reviews.


Now, the first thing to realize is that a "usability review" is different than a "grouse session". I was once doing a demo for an internal tool my team was working on, and after I'd showed how the tool worked, during the Q&A period one guy spoke up to say, "Boy, does that interface ever look like it was designed by a programmer."

"Interesting," I said. "How would you improve it?"

"Oh, you know. It just doesn't look as sharp as it could."

Well, yes. Nothing ever does; but it wasn't too helpful to tell me that. So, when you do a review, you have to be specific.

But how can you be specific about a UI? A UI is just a UI, right? It either looks good or it doesn't.

Not at all! There are lots of basic principles of design that the people who make web sites for a living know about. Even if your organization really is full of web pages designed by programmers, there's no harm in teaching the programmers some basic principles of design. I have a couple of books on that subject, one by Mr. Constantine himself, which I didn't even realize until I'd gone in to the session. But the organization or team should probably lay down the fundamental precepts of design that they want to follow. The usability defects will be easier to objectively identify with that list in mind. Some examples of good design principles are: Availability, Feedback, Structure, Reuse, Tolerance, Simplicity. Check one of the books for some guides as to the specifics, but a usability defect violates one of these principles, or you could also say it is a probable cause of user delay and confusion. But it's not a usability defect if you just don't think it looks good!

So here's how you prepare for a usability review: First, organize a few use cases. You may already have them as part of your project, or you may just have to make some up. What you'll be doing is telling the users what they're trying to accomplish.

Then, get the folks together. At a minimum, you should probably have:

• A leader, to make sure everything moves along smoothly;
• A notetaker;
• A Continuity Reviewer. This is someone who is reviewing the UI specifically to make sure it is consistent with overall project guidelines, and with the other pages in the project.
• Users - people who will attempt to use the page. They can be actual customers; agile-style customers; or just people who were walking down the hall at the wrong time.
• A Designated Driver. This is someone who will perform actual mouse clicks or typing at the request of the users. This will depend on the exact situation - do you have a real application, or just some mockups? Do you have a big meeting room and a lot of users, or not? If not, the Designated Driver might as well just be the user.
• Developers/Designers. Developers and designers who worked on the page must never explain or defend design, argue with users, or promise anything. They may only find problems. Users do not count as problems.

It's an important point for reviewing anything that if a reviewer doesn't find problems, he's not doing his job. I always have to remind myself of that. But the people who worked on the application; the programmers, the designers, the developers; they will always be able to give a reason for why it works the way it does. Don't listen! Mr. Constantine suggested a "virtual air horn" - you get to pretend to be a big truck and blow the horn to get people out of the way. You must blow the virtual air horn whenever excuses, explanations, or rationalizations are made.
Next, have the users go through the use case or scenario you've designed. Introduce the scenario with an overview of context and user motivation. Read one step of the scenario at a time, and ask the users what they would do next. Users take lead in proposing actions. Never guide or prompt users! Help is limited to simple description or clarification. If the user has to ask for help, you've automatically got a usability defect.
For each defect that you find, the notetaker should note:

1. The feature or function that the defect is in;
2. The location; which web page it is or a screenshot of the GUI
3. Which design principle is being violated
4. A short description of the problem
5. The estimated severity of the problem. (nominal, minor, major, critical )

Ideally, these would be on a form the notetaker would be able to fill out.
You should probably allow one to three hours for the review. So that's it! Get out there and say goodbye to applications that look like they were designed by programmers!

Web Application Risk Modeling

"Reverse" model - take the business case of the system and work down to threats.

A threat is not a vulnerability. A threat is what someone might try to do to your system; a vulnerability is how they would do it successfully

What risk drivers are there?

Application overview: Documentation drill; models; dataflow
Decompose application: break it down into well-defined "chunks".

Identify threats against the security objectives

Identify vulnerabilities "Vulnerability Assessments"

A threat model helps you to define, categorize, and prioritize vulnerabilities

Make sure to fix vulnerabilities, not exploits - understand all nuances, attack potential, exploit paths

STRIDE / DREAD

Other factors:
Ease of use, mitigants, timing, visibility,
monitorability (can you watch people doing stuff?),
forensics,
access required( even for internal apps, what are the chances of a bad guy infiltrating? )

XSS: Take user-inputted data and display it back without filtering. Nuances to XSS (Reflective Script Attack, Persistent Private Vectors)
POST based attack would not show up in server logs

xUnit Test Patterns and Smells

This comes from a really good session by Gerard Meszaros on Test Patterns at SD Best Practices 2007.

Here's my history on test-driven develoment: Back in the nineties, I first read Martin Fowler's Refactoring. I thought it was a good idea, and attempted several refactorings on the code base I was working on, with good success. I think it was one of the better-coded applications to come out of that company. But I was always annoyed, because the instructions for the refactoring would always say something like, make your changes, and test. Testing is hard, man! Especially when you're testing a bit of the application that takes two minutes to get to from application launch and relying on a Direct3D driver to do the right thing.

So I added refactoring to my arsenal but didn't think too much more about it, until about five years ago, when I ran across an article on TDD in, I think, Dr. Dobbs, but it may not have been. The article mentioned some ideas about testing and mock objects, which turned out to be exactly what I needed for the project I was working on then, which was a business-level client API with a wrapper lib for calls to the server - the ideal thing for a mock. I played with it for a while, and it worked beautifully! Pretty soon I presented a proposal for moving to TDD to the team I was working with.

There were a couple of quotes that I put in my presentation (probably from the magazine article) that I really liked:

• Tests must be easy to run. If they aren't, people won't run them.
• Tests must be easy to write. If they aren't, people won't write them.

This session was all about the second quote.

The problem is, tests are easy to skip. Comment out. Ignore. If you do that, your code isn't being tested. But the client doesn't care about that...at least in the beginning. Later on, if your code isn't being tested, bugs will start to crop up. You'll make a change in one area that you never in a million years thought would affect this bit of code over there. But it does, and you've introduced a bug. The client will sure care about that! So you really have to put the effort in to write tests.

But at the same time, you're selling the production code, not the tests. If your team is spending more time on the tests than on the code itself, your velocity is sure to suffer.

So what's the solution? Go back and look at the second quote again. Tests must be easy to write. How do we make them that way?

The first thing to notice is that your objectives for test code are probably going to be a little different than for the production code. For example, execution speed is crucial for production code. You can't have your users twiddling their thumbs while they wait for your web page to load. But for test code, not so much. Go ahead and add ten seconds worth of tests to your build; think anyone will notice? Or, add four hours worth of tests. Sounds good! Just make sure to run them overnight when no one needs to watch them.

On the other hand, is simplicity important for production code? Well...it can't hurt, of course. The smaller and cleaner you can get the code, the better. But sometimes there's nothing you can do about it; you have to add that cache for speed; or denormalize the database so you don't have to make calls across a dozen tables. But for test code? Let's say it again: Tests must be easy to write.

What else? Is correctness important for production code? Of course...but users will put up with small bugs. But correct test code is an absolute requirement. If you don't have the tests right, you'll be writing incorrect production code to satisfy the bad tests. What about flexibility? Code should be flexible, right? Not really, not test code. In fact, there will probably be enough hard-coded test values to make it hardly flexible at all.

This is getting long. I'll add more later.

Software in the large

Here are my initial notes on the Jutta Eckstein presentation on scaling agile development across large teams. Cleanup may follow :)

Scrum of Scrums
Crystal

Iteration Duration: larger the team, shorter the development cycle
per week, count on a half day of retrospective (two week cycle = 1 full day retrospective)

Expectation: plan/develop/deliver.
Difficult - activity-oriented planning or component-oriented planning?
Therefore: Result-oriented planning. Focus on the features! Comes back to the Agile Manifesto: Our highest priority is to satisfy the customer.

Plan for accomplishing a valuable feature: integration, test, documentation.
A feature is a brief statement of functionality, from the user's perspective
How does one deal with architecture issues?
A feature produces a measurable result.

Iterations are steered by features, but defined by tasks

Tracking tools: PPTS, TRAC
Someone also mentions they use Sharepoint
Or just three checkboxes: working on it, untested, done done
Tools support communication, not replace it

Release Planning

Iteration review (Demo)
Present software, recognize & extract best practices, learn from failure

Measurement: Acceptance tests, planned functionality, is the product owner satisfied?

Retrospective after every iteration. Likely problem that people try to make large-scale changes

- Cross-functional or feature teams
- A large project might have tech teams; the customer of a tech team is a feature team

An ideal team is self-organized; this ensures whole features and good knowledge sharing. Managers must provide environment allowing teams to gel. This is like my ACG posts from a few months ago.

Trust

Agile development is a trouble detector. Bad news is also good news. Integration of departments (Projects are customers) Close customer relationship ensures rapid feedback.

Discussion of implementing practices a few at a time. Ping-pong implementation!

Synchronization: Face-to-face is preferred. Sync across subteams daily (Scrum of scrums). If your team is self-organizing how does that work?

Communication via wiki

Just one "Chief architect" - pulls the strings, makes technical decisions, "guiding light". Relationship of chief architect and customer?

Starting: take baby steps. Start small. Use skilled people. Develop a few features and make sure to do iteration retrospectives. Grow slowly.

Don't finalize architecture before growing team; use retrospectives. Domain teams must formulate new requirements. (But you might have to finalize to eliminate fear...or at least say it's finalized!).

Avoid hot technology. A large project has enough problems on its own without trying to train developers on something new at the same time.

Refactoring: technical excellence is doubly important. If a developer sees a needed refactoring on another team, they have to point it out to them.

Large projects may have exponentially greater test time. 10% of dev effort for integration/build. (If something is difficult, do it over and over until it's not difficult any more.)
Q: Special iterations for integration? A: no
Nor a special integration team; rather people from each team who specialize in integrating

Reviews:
Special review team. People should jump around between teams, and be on a team strictly for the purpose of reviewing the code. Everyone should do this.

Knowledge transfer (via Daily Scrum and pair programming). Scrum master ensures the process; product owner ensures business value).

Q: Agility in a distributed environment. A:

Could not load type 'Global'

The comments on Harish's blog entry from two years ago give a lot of different solutions to the 'Could not load type "Global" ' problem that you sometimes get in ASP.Net. My solution to the issue was an interesting twist on one of those answers.

I had recently upgraded an application from ASP.Net 1.1 to ASP.NET 2.0, but to keep supporting old versions of the application, I branched it off in Subversion. To make sure the old version still worked, I checked out the old code into a new folder, then I went into IIS and simply moved the location of some virtual directories to point to the old code. It all worked and forgot about it.

Until later, when I came back to make some changes to the new application, started it up and got the message:

Parser Error Message: Could not load type 'WebApplication1.Global'.Source Error: Line 1: <%@ Application Codebehind="Global.asax.cs" Inherits="'WebApplication1.Global'" %>

I couldn't make heads or tails of it, but a web search led me to Harish's post and lots of different answers, several of which I tried, but none of which worked. One suggestion was to make sure the application was set in IIS to use ASP.Net 2.0 rather than 1.1, and even to set it to 1.1, click Apply, set it to 2.0 again and click Apply, just to make sure it took. Another was to make sure the application was compiled. If there's no assembly built for the application, it won't load.
I checked the ASP version, and it was indeed set to 2.0; I said, "Duh!" to compiling, but made sure there was an assembly in the bin directory, and there was; so I was at a dead end.

But I'm sure by now you see where this is going. As I opened up IIS to check the ASP.Net version configuration again, I happened to glance down at the local path for the virtual directory on that tab. And what did I see? The directory was still pointing at the path to the branched directory; a perfectly legal application, but one that was built using ASP.Net 1.1, and also had been cleaned sometime in the not-too-distant past. So the version I had configured in IIS was neither compiled, nor a 2.0 application! No wonder the error came up.

So I have an additional solution for this problem. Check your virtual directory location and make sure it's pointing to the application you're expecting it to be.

Pair Programming with VNC

Dietrich Kappe writes on the Agile Ajax blog on surmounting the difficulties of pair programming when part of your team is offshore. Interesting stuff, but Dietrich, you also made the offhand comment that Test Driven Development is one of your commandments as well. What's your process for writing Ajax unit tests, and if you're not always doing continuous integration, how do you know your tests are always passing? I'd be curious to know!