Sports and power ranking systems

I've had an interest for a long time in the science of sports team rankings, for various reasons, which was forcibly brought to mind when I was filling out my tournament bracket. I'm always extremely mediocre in such game-picking contests, and when one blogger whom I respect said something similar, I started to think about how rankings could be done for college basketball. I looked a little closer at the Pomeroy Rankings, which seem pretty nice, although if Ken reveals his exact formula for creating them, I couldn't determine it. IMO a ranking system can't be taken seriously if the method that is used isn't known. Take Jeff Sagarin: Everyone always prints his rankings up very seriously, but we don't know what he's doing, so he might as well be making them up and just pretending it's math.

But Ken has something much more valuable on his site than a ranking system: a game database. For the most part, this information is not available in any easy-to-get-at-form, so if you want to create the rankings, you have to get down and do the data entry every year, which is why I've never created any system that lasted more than a year. But now, with Ken's files, maybe something useful could be done.

So I did a little research, thinking that the most effective system probably was going to be some kind of balance between a single-game Pythagorean expectation and strength of the opponent, repeating until the numbers converged. I'm sure I read a paper about that some years ago, but I can't find it now. Instead, I found this, a technique which doesn't take into account the scores at all!

But it's interesting, because it's based on the age-old theory of game commutativity; to wit: my team beat team X and team X beat your team, so my team is better than yours. Yah. It's a principle that's been widely derided for years, and people make hobbies out of finding weird cycles of games proving that Prairie View A&M is really better than Michigan after all. But there's obviously a kernel of truth in it. The paper goes into a lot of detail about setting up the graphs and putting weights on things and, you know, math, but really the principle is pretty simple. It works like this:

For each game that my team wins, it gets partial credit for each win the team it beat has.
For each game that my team loses, it gets partial debit for each loss the team it lost to has.

That's it. The questions are, do you want to go deeper and credit my team for a third or fourth level, and just how much credit do you give for each "indirect win"? The second question is easier for our purposes, because the authors of the paper do a lot more of that math stuff and come up with a simple equation for us:

Let k equal the average number of games played by each team.
The credit is (2k) / ((k^2) - k ).

For a third level, you'd square the credit, etc. But do you want to do the third level? Say the credit is .1, or 10% of a win. For the third level the credit would be .01, which doesn't seem like much, but you're talking quite a few games, too. So I'm going to have to use Ken's game database and do some research on this. Any code I create will be open-source, of course. I won't be able to do anything useful before this year's games start, but next year, watch out!

Generating classes from XML in .Net

The Oracle version of SQL has some nice keywords for returning your data in an XML format. (I suppose the other servers do too, but I've not used that feature.) When I get the XML back, I want to turn the XML into a set of business objects for easy serialization. XSD is the tool for that. Write the XML to a file, run XSD on it to generate a schema, then run XSD /c to generate the C# class file, and you've got a nice class. You can muck around with the XMLElement and XMLAttribute attributes to create nice field names, and 30 seconds to put together a static Get() method that returns a class from the XML.

Except it didn't work. The serializer threw a File Not Found error. When the XMLSerializer class has a new type it needs to serialize, it just generates the code on-the-fly and throws it into a new assembly with a name like olkdzxc.dll, and returns the class from it; but when I called the serializer, it told me that olkdzxc.dll wasn't found. Very mysterious.

Luckily, I remembered Chris Sells' old tool that was made for debugging exactly this problem, XMLSerializerPreCompiler, which lets you see the compiler errors that occur while the code is being serialized, and one of those led me to the problem: When generating the class code for an array of objects, XSD was adding an extra set of brackets in. So instead of having a class member myFoo[], I had a member myFoo[][]. Why did XSD do this? I have a hard time believing it's just a silly bug. I'd love to hear if anyone knows.

Online and offline communities

I wrote earlier about creating online communities around business-to-business applications, with a vague promise to add more, but didn't. What rejuvenated my interest in the topic was going to a lecture and buying a book by one Joseph Myers, who lectures on creating small groups for churches. Churches generally don't like to be impersonal - it kind of misses the point - but there's no real alternative once the congregation grows beyond 100 or so people, so they like to subdivide into smaller groups so everyone has a group they can be comfortable with. What Joe Myers pointed out, as I understood it, is that it's very hard to create this sort of a group from the outside - intimacy generally arises from a group of people who like to be around each other. So, if you are a corporate executive, or a church leader, who is tasked with creating a community, how do you go about it? On the one hand, it's our job to create these communites, on the other, we know that they're not created, they just appear.

So the recommendation, for church leaders at least, is to define more exactly what groups already exist in the church. Church leaders want every small group to provide intimacy, but that's really only one way to relate to a group: the group can be more of a public group, or more of a social group, or just a personal group, and churches can take advantage of knowing how these groups relate to each other to encourage more fellowship in the church.

Is it applicable for online groups? I'm not sure. Here's the issue: at least if you're working with a church congregation, you can call a meeting, bring everyone in, discuss the issues, maybe figure out what the existing groups are and what they're doing. You can't do that online. Maybe the best thing someone tasked with creating an online group can do is simply to monitor the group, or groups, and make sure that the company is willing to go wherever the group takes them. Seems obvious, but is it? Check out Yahoo's handling of Flickr accounts, or Facebook's decision to allow non-college students to join. Or check out a lot of different online forums that die because people thought they were cool at first, but then they never changed again and everyone left for more responsive pastures.

I don't know the answers. But it's an interesting bunch of questions.

Quality of Local Political Blogs: Compare and Contrast

I've been writing various articles for Bloomingpedia in the last month or so, in an effort not so much to improve that site as to understand better the town in which I live. One of the keys to understanding a city, I think, is to gather a lot of different perspectives from a lot of different individuals with a stake in the matter. Take Indianapolis, for example: there are a lot of places you can go to get an impression of how the city is doing. Ruth Holladay; Matt Tully. Taking Down Words; Indy Undercover. You have to gather them all together before you can make a critical analysis of what's really going on; but they're there, that's the important thing.

Or you could read the Indianapolis Star. But I don't have much trust in the Main Stream Media. Their goal never seems to be so much the truth as it is finding someone who disagrees, no matter how foolish or inane that person may be, and unless you already know the subject matter pretty well, you can't tell from the way the article is written which is the inane perspective and which is sensible. So that leads you back to blogs.

Here are four local politicians who have been on my mind lately: Marty Hawk, Dave Rollo, Scott Tibbs, Sophia Travis. How easy is it to get their perspectives on local issues?

Far and away the best online writer in this group is Sophia Travis. If you just looked at the MSM, you wouldn't think much of her except that she's a little flaky (an accordion player with political aspirations? Weird!) But when you read her blog, not only is she talking about the tough political issues, but she's following up on comments people leave; leaving comments on other local blogs; sending in questions to local online chats; really being a part of the conversation about what Monroe County is, and what it should be. It would be great if every politician had an online presence like Sophia's.

Second best is Scott Tibbs. I actually started this post thinking about what I don't like about Scott's blog: there's no real comment area on it, just a link to a bulletin board, which I assume is also run by him, and which you have to register on before you can comment. He says that's to avoid spammers, but obviously a lot of bloggers manage to allow real comments without going to that extreme. But the point is, he writes, and discusses, and allows discussion of his views in some form. So I can't take too much umbrage, especially compared to:

Dave Rollo. He's got a web page; it's a start. The page is very static; the main page has a "last updated" date on it, but there's nothing to find what was there before. There's only a few paragraphs discussing his views, and there's no way to leave public comments, and if he's ever left a comment online I haven't seen it. Start a blog, Dave. He did participate in an online chat recently, and having a web page puts him ahead of:

Marty Hawk. Not much to say here, because I really couldn't find out anything. She gets quoted in the local paper from time to time, and you can go read the minutes of the Monroe Council meetings and find some things she said. But right now, the number 2 hit on Google when you search for her name is the article I wrote on her last week. So we really don't know too much about her at all. It leaves me defining her, rather than having her defining herself. If that's what she wants, then that's fine.

So that's where we are in online local politics in Bloomington. It's a start. But I wish there were a lot more politicians in the conversation.

Bookplates

(Hey, this is my 200th blog post! And it only took me three years!)



One principle of agile development that doesn't get a lot of attention is Sitting Together. The point of the principle is simple: agility requires communication, and there's no faster communication than shouting over your shoulder to the guy behind you! I think it's a bit overblown; communication is hugely important, but with the advent of instant messaging, not only do you know that Dan down the hall is sitting at his desk, but you even know that Mike down in Dallas is, and they're just as likely to respond to your ten-second query as Jennifer two desks away is. The participants have to be in pretty close time zones, though; Suresh in India just isn't gonna respond to your IM no matter how many times you check his status during the working day!



In my new company we sit together, which is something I've never done anywhere else. I've found that one disadvantage is that my desk doesn't have space by half for my programming library, which I like to keep at the office for easier reference. (Okay, so I haven't referred to the Differential Equations textbook since I left the videogame industry. Nevertheless.) So I'm taking over a couple of shelves nearby, but instead of just writing my name in all my books, I thought it would be more fun to make bookplates for them. Here's the design I made:

I'm no graphic designer, but I thought it was OK. If you want to modify it for your own use, feel free; I've made a Word template available for use with the Avery labels that come six to a page; you can get it here, or download the Avery bookplate for the four-a-page labels. Hey, my favorite book site LibraryThing, why don't you provide some of these? I'm sure there are dozens of people who can do better!

Government RSS Feeds

Sophia Travis is a Monroe County Council member - and, remarkably, one who is sophisticated enough to have her own blog. On one post, she asks what information we'd like to see on the Monroe County web pages. I would like to see an RSS feed. Here's why: Government business does not lend itself well to the regular web page format. The people's business does; the most important thing for the web site will always be ways to contact government officials; how to apply for permits; pay parking tickets; vote, etc. But government business consists mostly of a neverending string of public meetings, each one with an agenda beforehand and minutes afterwards. The best way to present a stream of information like that is with a feed. For example, I've already created a feed for the Council meetings using the very nice, if complex, Feed43 service. My feed will do the job, giving me an update through my feed reader whenever a new meeting agenda or minutes are posted, but it's pretty content-free, as the feed can't do much except monitor each row of the table of meetings on the page. But suppose the county tech services people set up an easy way to post updates using TypePad or Blogger - suddenly it's easy for them to update the site and there's a good description of the update in the feed. Then, perhaps, it could be expanded using the same feed, to give information on other public meetings, notice of events the council members are participating in, and any other kind of information that has a time element. I'm thinking this is actually a time saver, at least for that one poor soul whose responsibility it is to go in and edit the HTML table on the page whenever new meeting minutes are available!

What would you like to see on your local government web site?

Change is good

I've accepted a new job with Envisage Technologies, a small software company in Bloomington. I'm excited about it as it's a company with a firm interest in agile principles:

Do ideas by the Gang of Four, Steve McConnell, Martin Fowler, Tom DeMarco and Kent Beck resonate with you? Join an experienced team of developers in an Agile environment...

So I'm no longer working in Indianapolis for the first time in more than ten years - I'm not sure what I'm going to do with all the extra time!

(I've also set up a LinkedIn account as per Guy Kawasaki's suggestion. Drop me a line if you want to connect to me.)

Prius anti-skid props

The problem with having six inches of snow dumped on us is that the hill that leads to our house has a slope that isn't quite a vertical wall, but that's pretty close. So as I was heading home my wife assured me she'd seen the plow go by, and I decided to take the hill, which in good weather would be five minutes, as opposed to going the long way round and taking half an hour.

Up I started, accelerating to about 25 MPH and getting at least 30 or 40 yards before realizing that the plow hadn't been by recently enough to make a difference. It was easily the worst snow I'd ever tackled on the hill before, and it's not fun having to back down that slope, let me tell you. Especially with the literal vertical drop on the side that sends you ten feet straight down before the drop is conveniently stopped by a tree.

But here's what the Prius does, straight from the brochure:

Motor Traction Control (TRC) – TRC uses sensors which automatically apply the brake to any slipping wheel while delivering more power to the wheels with greater traction.
Vehicle Stability Control (VSC)* – VSC senses oversteer (tail slide) and understeer (nose pushing forward), and managing the power delivered to each wheel.

It was a beautiful thing. I kept the accelerator right around 25 and the car took over from there. It never slipped sideways, never fishtailed, and actually applied acceleration to the wheels in bursts of a couple of hundred milliseconds at a time, followed by coasting to grab what little traction it could, and then accelerating again, and I was at the top of the hill as nice as pie. I only felt guilty for not stopping the cars I passed and telling them, "Your car got TRC? Got VSC? Then DON'T try the hill tonight! Just because my car can do it doesn't mean yours can!" What a beautifully engineered vehicle.

WiX installer and Error 2708 (No entries found in the file table)

I had to add different versions of a file to my installer today. Seemed easy enough to do - the files were available in the right places and everything, so I added a version number on to the old file ID, added a new file with the new version number, and called it a day, right?

Not so fast. Compile up the install and run it:

Error 2708: No entries found in the file table.

Say what? Must have been a file system glitch. Open up the MSI with Orca and check the file table; well, yes, it has lots of entries, no trouble there. What's going on here?

Buried deep in the search results for the error code I found this page. The comment from Jane D pointed out that she'd seen this error while having problems with the Duplicate File table rather than the File table - and that jogged my memory. In a separate component I had a CopyFile element that was pointing to my file, and it still had the old file ID reference, now orphaned. Update the reference, recompile, and bingo. Working install.

I see this as a bug in the WiX linker: why did it build the MSI with this unresolved reference? I'll have to post something to the mailing list at some point.

Credibility redux

Y'know, I'm not affiliated with Microsoft at all, although I own some stock and use their products happily. So I'd like to think that when I say something like, this Microsoft employee has credibility, it's because the employee actually has credibility and not because I'm biased or brainwashed towards the Borg.

I've been a loyal subscriber of Dare Obasanjo for at least a couple of years now, and a happy user of RSS Bandit, although I'm now evolving a bit into Google Reader for its mobile capabilities. So when I read his article about changing Wikipedia I didn't think much about it; mildly interesting but not a big deal, and his changes in the TechCrunch entry certainly deserved reverting under the Wikipedia "No experimenting" clause. But Michael Arrington's reaction was out of line:

A Microsoft employee, who took issue with this blog post, vandalized the TechCrunch Wikipedia entry and wrote about it on his blog.

That is a misuse of the word vandalized by any stretch of the imagination. Dare added maybe a couple of sentences with a dry, unemotional tone. He put up an apology in the comments, too, but in two or three comments (which have now disappeared) Arrington repeated the vandalism charge, and he's showing no signs of backing down. IMO, there is a serious credibility gap in repeating an emotionally charged word like that in response to some rather minor issues. I'd never heard of Arrington before, or read TechCrunch. This little flap doesn't make me want to, either. Michael Arrington joins Andrew Orlowski in my credibility book.

B2B 2.0

Chris has left LibraryThing. I don't know him, but apparently he's done some really good work for one of my favorite Web 2.0 plays; a social site based around book collecting.

There are lots of definitions of Web 2.0, but at least one of the principles that seems to define it is "Online Community". Flickr, YouTube, Yahoo! Answers. Online communities have been around since the beginning, of course, at first through mailing lists and NNTP servers, later through applications and, eventually, web sites. When we at Sunstorm were working on a version of Deer Hunter that was going to have a multiplayer mode - we had only the vaguest idea how that might work - I went to a seminar at the Game Developer's Conference on the topic of building online communities. We did a little work towards it; our web site ran some decent forum software, but in 1998 the Deer Hunter target market did not actually overlap with people who spent a lot of time online, which was hampering.

At least we had a good size target market. Combine the lush outdoor scenery of Deer Hunter 3 with a visionary concept of online communication, and we might have had our own version of Second Life on our hands, five years before anyone else. But of course, we didn't have the vision thing. It's still the hardest part of launching a consumer oriented web site. Wal-Mart tried it. Xanga was hip for a while. Not much there there, now.

But what about an online community as part of a B2B play? Not a corporate MySpace, but a self-selecting group made up of users of your product. If your target is geeks you might have a leg up here; Kinook has a nice online forum. Axosoft has forums and bloggers. The forum we put up at Interactive Intelligence seems to be buzzing along nicely. When I was there the customer base was very technical; that may be less so as their customer base has grown. But I think an actual, product-based online community is very workable for a business-to-business company. More later.

The blogging split

I got into a discussion with someone concerning the blogosphere and its effect on corporations today. Here's something I hadn't really thought about in a couple of years: when was the last time someone was dooced? It's been a while, I think; or at least when it does happen it's not much of a story and the mainstream media doesn't pick it up.

So what happened? Did everyone just sort of "get it" ? I would say more that the world is sort of partitioning itself off now. On the corporate side, corporations are splitting into sort of "New Media" companies, Microsoft and Sun, where bloggers are allowed almost free rein, and "Old Media" companies, Wal-Mart say, or GM, where they feel it's very important that the company try to keep absolute control of the image of the company and don't allow their employees much say. That's not to say there isn't crossover; I understand one Microsoft division wanted Robert Scoble fired after he said something critical about the company, while GM actually has a blog...a rather corporate-oriented one, to be sure, but it does allow comments and they don't appear to censor them for criticizing the company.

On the other hand bloggers, or better I should say people, are splitting off as well. You see a lot of blogs around where someone started the blog, posted a few things, then apparently dropped off the face of the earth. Or possibly they write an article once a month or so apologizing for not blogging more and promising to do better from now on. Hey, blogging is hard, and most of us aren't getting paid for it. I've been known to go a month or two without posting. So there's more of a split between people who blog and people who read.

So I suspect what's happening is that people who blog, are moving over to work for companies who support blogging! Maybe not a momentous insight, but I can't think of anyone else who's come out and said it. People who don't blog, can stick around with the companies that are trying so hard to control their messages. That's why, I suspect, that you haven't heard much noise about doocing recently. People have sorted out where they belong; companies have clearer policies about what they expect and employees have a clearer understanding of what they're looking for.

(If they don't, I guess they'll have to buy Shel Israel's new book to clarify things.)

Building a cathedral

A traveler visited a city where many stone cutters were working. Approaching several, he asked the same question: "What are you doing?" The first stonecutter he met replied, "I'm cutting stone. It's dull work, but it pays the bills." A second stonecutter responded, "I'm the best stone cutter in the land. Look at the smoothness of this stone, how perfect the edges are." A third pointed to a foundation several yards away, and said, "I'm building a cathedral."

Thanks to Grady Booch and Joe Marasco for the story!

New Technology High School

This High School is a model for high tech high schools that's being pushed by the Gates Foundation and probably some other places, and they're thinking about trying one out in Bloomington, or at least they've gotten 50 G's in order to study the possibility. Some of the money went to bringing in a speaker to discuss the possibility, so along with a couple of hundred other people, I went to check it out. My initial reaction is, Groovy. I hated high school, and it's only ten years from now that my son will start it. I'd like him to have some choices about where to go, and this model seems pretty nice. They want to keep it to 400 students, as opposed to the thousand or so at each Bloomington high school now. I've commented elsewhere about not really understanding the Indiana charter school system; I suppose this would be one of those.

I'm hearing some contradictory things about the school, though. For example, a questioner asked last night about the per-student cost of the school. The response was that the school doesn't get any more from the state than any other school would get, and that technology was the biggest expense. But the little handout we got actually says, small school and class size allows students to take responsibility for their own learning...So I wonder which it is. I'd guess that any school would find that graduation rates would inversely correlate to class size. Also, two separate articles in the paper (subscription required) tell us that the school (a) caters to students in the job market, and (b) most graduates go on to higher education. What the heck does that mean?

The speaker explained a little bit of what the school was about; all very nice; focus on communication skills and working as a team, computers for everyone, community internships. I think you can have two kinds of high schools: the kind where kids are motivated and enthusiastic about doing stuff, and the kind where the kids are biding their time until they can get out and go do something else. When you have the first kind, the students are going to be self-selecting - they have to want to go to the school. This is why I think charter schools and school choice are good ideas. So for that reason alone I think this school would be a good idea.

But the audience had a lot of good questions; some sublime; some ridiculous; all very practical. The inevitable "What about sports?" question was asked, which of course really means, "What if my kid wants to go there but he's also a basketball star?" The responder didn't really pick up on that dynamic, mentioning that the schools in California play Ultimate Frisbee against each other. Yeah, great. But the local guy did mention that allowing the students to play on the big school teams was a possibility.

A lot of the questions made me think, though, that either by state law or by educator attitudes, the school system isn't really ready to shift paradigms. I don't necessarily blame them; it's not an easy thing to do. But there were questions about honors degrees and demographics. The California panelist pointed out that an honors degree is a pretty divisive thing, and how can you teach teamwork in that sort of environment? The local panelist said that he thought the demographics would have to mirror those for the local high schools, so this school would have the same proportion of special needs students, minorities, gifteds, etc. I don't see how they can do that and still have the students be self-selecting, not to mention I find it extremely irritating when people are classified into "black", "poor", "special ed" or groups, even when the goal is to create balance.

So there's plenty to think about still. But I hope they do it. And if I'm still around town in ten years, I'll probably be pushing my kid to go there. If you see a chance, take it.

The Guerrilla Guide to Interviewing

Joel's latest article on interviewing is up. It makes some good points, although he continues with the down-to-the-metal idiosyncracy that I posted about last year. But here are the things that I thought were really good points:

• Hire/No-Hire . Make a decision. If you don't know, the answer is No Hire. I've run into this before when interviewing an entry-level guy for a position that required more skills than that. We recommended he be hired for Support instead. I'm not sure that that wasn't the right decision, but as a principle I like this one.
• You want people who are smart, and who get things done. Joel describes people who fail at one or the other, and I think I've worked with most of them before.
• A programmer should understand pointers, and recursion. Joel comments that a lot of people are coming out of school without learning a language that requires pointers, which is a problem. Less so with recursion. He says that pointers are an aptitude rather than a skill.

At the end he says, confidently,

If your resume and phone-screening process is working, you’ll probably have about 20% hires in the live interview.

True at FogBugz, no doubt. I've not really seen it here in Indianapolis, where the local talent pool is so small. But you never know, we might get lucky!

Want a job at an up-and-coming medical imaging company? Drop me a line!

Bloggers are people too

Ordinarily for me, reading and writing blogs is an intellectual exercise. I'm more comfortable and interested in discussing software processes, languages, testing than I am with the emotional appeal of the typical network news sob story. So when worlds, and cars, collide, I find it affects me especially. I envision my own five-year old son coming home from his karate class and I feel like crying. Good luck to you, Nick and Josh.

IQAA: Regression Testing

Dr. Hanna's Practice #8: Perform regression testing that is based on impact analysis. The first practice in the list that you can't just nod your head at, since you have to have a good idea of what regression testing is (I do) and what impact analysis is (I didn't.) But I did like Practice #7: Testers should attend design and code reviews. It's not something I had heard before, but it's obviously a good idea if you are interested in faciliating communication within your company.

So what should a tester do at a code review? Primarily they will want to come up with test ideas; examine the code paths; ask how each one can be exercised. But also they can ask a very fundamental question: What other parts of the code is this project going to affect? This is an impact analysis. If I remember correctly, it was recommended that this analysis be done formally, as in developers have to write up a statement or report analyzing what other parts of the product will be affected. Not a bad idea, but probably not for smaller companies like Prosolv.

So based on the Impact Analysis, testers should be able to come up with a set of requirements that need to be retested, and there's your regression suite. Of course, every build that goes to testing should be tested on the critical path (or as I prefer, the "Happy Path"). Dr. Hanna suggested a 90% pass goal, but I'm not sure why that should be. Some tests will be showstoppers, others will be...well, whatever. I suppose if you have more than 10% "whatevers" failing, you've got an issue, though.

Just a couple of other notes:

- Regression testing doesn't do any good if you do it at the beginning of a project - it is certainly to be hoped that there will be few failures then!

- Impact analysis is also necessary when a requirement is changed. Go to a developer if necessary!

- Which led to the question, what if the developer doesn't know? Dr. Hanna's response: Find new developers ;)

IQAA: Integration Testing

Dr. Hanna's Practice #3: Test for both functional and quality requirements. I would have thought state charts and truth tables were familiar to everyone, but I think the typical Indianapolis tester has a lot less experience than, say, one in San Jose, so there were a lot of people who they weren't familiar to. But Dr. Hanna had some good advice on turning a requirement, written in English, into a model by splitting it up into actions and results. He took a typical requirement statement, teased four predicates and four consequences out of it, and showed the truth table that it resulted in, with the 16 possible states of the predicates and the expected consequences of each. I was a bit itchy here, because I always feel that you can go through and test your application like this, but then go through and test a separate bit of your application which contradicts it. To help my understanding, I went on the second day to a talk on integrations testing, which I thought would more or less cover my confusion. You've got one requirement, you've got another requirement, testing them both is integration testing, right?

Well, no. Integration testing is the actual bit where you take two components of the system and make sure they talk to each other properly. Testing the input/output of one component is mostly a unit test, since they usually are easily testable and verifiable based on the automated testing that should have been written by the developer. But you need integration testing to avoid the "operation was successful but the patient died" phenomenon, where the interface of component A is not clearly understood by the developer of component B, so he writes and tests a very nice component that doesn't do at all what component A expects.

But with that clarification, I guess I see the real issue: the thing that is worrying me are two contradictory requirements. Given a clear requirements document, it is no longer the tester's problem, and it is no longer the developer's problem. It's a business problem, and someone with knowledge of the problem domain is required to clarify the contradictory requirements, which allows us to update the requirements doc, and guess what - now the testers can redesign their test plan and the developers can redo their code.

Here's a list of books recommended at the conference.

IQAA: Changing Requirements

The whole conference I was at this week for me revolved around the requirements management process. Partly because many companies I've worked at had trouble with this part of the process; that is, they followed the process of creating a requirements document, then they shut it away in a drawer and never look at it again while developers and testers go along their merry way and code up a mishmash of the requirements, what they think the requirements might mean, and any customer requests that aren't too difficult and/or are from important customers.

But I think a main thrust of Dr. Hanna's talk was that the requirements document is very important. I'm used to this very static, dull requirements document, and so I kept wanting to raise my hand and say, "How can you do that when the requirements phase is already complete?" But I have to conclude that he doesn't think it is static at all, and that it has to be dynamic and updated continually. (It was interesting that he said several times that testing is a process, not a step in the process, but he never said requirements were too.)

The typical software company tends to communicate rather informally. Write up a vague requirements document, then have the developers implement it any ol' way that seems right. If they're good, or at least social, developers, they'll talk to customers or managers or somebody that can clarify the requirement. A lot of developers will just guess, though. (Combined with receiving fast feedback from a Customer, this is just fine, of course.) But this is why the developer/customer communications need to be with testers (in a typical software environment ) or part of the process (in a regulated environment or one with traceability requirements. When it is part of the process, the correct process, I think, is to modify the requirements doc based on the customer communication. This gives testing a chance to update their tests. Dr. Hanna came back many times to the diagram:

Requirement -> Test Scenario -> Test Case -> Script

So if the Requirements are up to date, the tests can be up to date as well.

I'm not sure that every attendee thought this was the emphasis, but I also went to a couple of talks on this topic.

IQAA: Quality enrichment conference

I've posted about the IQAA before, and I regret that I haven't been able to make it to their talks regularly. There is a bit of a disconnect between the organizers and the members, I think, because the organizers are creating a fair amount of high-quality content, and I'm not sure that the Indy software testing scene really is vibrant enough to appreciate it. Today I'm attending a free seminar given by Dr. Magdy Hanna of the IIST on Software Testing Discipline and Software Testing Management, and it's very good.

The intent was for Dr. Hanna to give two seminars, one in the morning more or less aimed at testers, and one in the afternoon aimed at test managers, but in practice they all sort of collapsed together. The majority of attendees were there for both sessions; which was good, because they ran together pretty much. Dr. Hanna is a good, knowledgable, and confident speaker, and when you have one of those you're guaranteed to run over. We got to hear a little more than half of the practices before lunch, and a couple more afterwards, so what was billed as the "afternoon session" started around 2:00. But it covered basically the remaining practices anyway, and around 3:15 he looked up, said, "How much time do we have left?" and burned through the rest of his slides as if they were a kaleidoscope :) I'll put together a few posts over the next few days on my impressions of the conference and speakers. I'm not going to summarize all of the practices he named; just some of things that made me think. For example,

Practice 1: Requirements are crucial, with the couple of subheaders: You can't test what you don't know, and Users will always change their minds, and this was the point when he went all Steve Yegge on us, and explained how he was opposed to the agile movement. Of course, as is usual in such cases, we find out that he's not actually opposed to the practices of agile, or at least many of them, but only to calling it agile, or something. (I've never been quite clear on what exactly the opposition is to).

I mention this in passing because it seemed to me that those two headers absolutely contradict each other. How do you know what to test, when the users are calling the developers daily with new requirements? But his overall point, I concluded, was that (a) requirements documents should be kept accurate and up-to-date, and (b) they should be your main avenue of communication between developers and testers. I had assumed, when he said he didn't approve of agility, that he wanted nice static requirements docs before testing ever started. This, of course, never happens in the real world. More later.