Ramblings of a software developer with a degree in bioinformatics. Agile development mixed with DNA sequencing - what could go wrong?
Tuesday, October 16, 2007
Bloomington Geek Dinner
But none for Bloomington, until now. Or even Indianapolis, as far as I know; if you're up in Indy feel free to come down - we'd love to have you.
It'll be at Max's Place downtown, starting around 7PM. See you there!
View Larger Map
Saturday, October 13, 2007
Indy Tech Fest
Big room has no wifi; smaller room seems to work pretty well. I feel very blind without a net connection. Have I said that before? :)
Session #1:
Stephen Fulcher, VS 2008
3.0 install only has WCF and WPF, everything else is the same as 2.0
Integration is tight to SQL Server. Other db's?
anonymous variables - new keyword var (not untyped, but takes on the type of the right-hand side)
lambda funcs,
extension methods: public bool IsBool( this string s )
Generate code metrics?
Q: Is var a breaking change? A: no, you can still use it is the name of a variable
XAML
Services (not web services, just services)
ServiceContract, OperationContract, DataContract attributes. - all specified as an interface
svcutil.exe generates client code and can resolve from existing DLL's
----
Session 2:
Mark Strawmeyer
C# tips & tricks
ctrl-alt-downarrow window list
ctrl-/ takes you to search menu
In the search window, type a ">" to go to a list of immediate-style commands
F12 is "go to definition"
Shift-F12 "Find all References", then use F8 to cycle through them
Code snippets - intellisense to generate code - type in the code and tab (to autocomplete), tab (to generate)
very powerful! prop, for, tryf
"Organize using" tools in VS2008
Unit testing support:
Test input from data source
unit test assemblies without source
Now part of VS2008 Pro
"Create Unit Test" option
Code Profiling is not part of Pro
---- Break for lunch. There are some lunchtime sessions but we just hung out in the lobby and watched the XBoxers.
Session 3:
Mark Strawmeyer
Ajax Tips
eh. Mostly just an overview of Ajax.net
Session 4:
Silverlight
Chad Campbell
Silverlight is the new name for WPF/E. Create a silverlight object using xaml. Add "Silverlight.CreateObjectEx()" to the HTML to use the plugin.
You can use .Net with silverlight 1.1. Silverlight is a browser-based plugin. It's cross-browser, cross-platform. What does that mean? They've written a plugin for Firefox as well as IE? Supports Mac and Linux and Safari as well. .Net framework used by Silverlight is specific to Silverlight.
Access the HTML DOM from managed code. Silverlight plugin will not be able to access the local file system.
Session 5
Dave Bost
WPF applications
Starts by demoing a Twitter feed on his page, created with Silverlight
Is it going to be the same presentation he did in Indy about a year ago? I don't think I blogged about that.
WF, WPF, WCF, Cardspace. WPF is .Net layer over DirectX.
Nice demo of an airport simulation
Expression is a set of tools for designers. Expression Web, Blend, Design, Media. Costs a bundle, though. Once you design the xaml, you can edit it in Visual Studio.
http://devcares.com in Indy every month.
Friday, September 21, 2007
Continuous Integration
Graphing the number of tests over time would seem to be a good idea
Sligo dashboard: classes, LOC, duplication, max complexity, tests run, line coverage, branch coverage, FindBugs violations; PMD Violations; Max Afferent; Max Efferent
Code reviews are good for high level details; use machines to do the low level detail finding
Code complexity tools: CCMEtrics, Vil
Code duplication detector: Simian
Dependency: NDepend
Coding Standards: FXCop
Thursday, September 20, 2007
How to work with an open source team
"Free as in Beer"
http://opensource.org/
Q: Don't people resent when companies take open source projects and make money off of them? A: More power to them! Many companies are using their employees to do open source. It's good PR for a company to have people who work on open source - give back to the community, attract high-power talent
Don't contribute unless you:
Know the project license
Get permission from your employer
Get legal review if needed
Can communicate clearly in the project language (usually English)
Oracle tried to strongarm Linux, got squashed, came back with offers to help. Good PR! (I'm not familiar with this story!)
Project currency is trust and respect. You don't start with any. Remember, if you're good, you don't have to point it out.
Q: How do you start gaining respect? A: Post to the mailing list, point out bugs AND fixes. Maybe someone will request a patch, provide it
Q: How does the code stay consistent and looking good? A: There are tools, or people who do work to make things consistent. Or, it doesn't :)
Q: How do you get non-coding contributions going (docs, images)? How do non-coders get cred? A: Projects should support people like tech writers, if they're good.
How to Gather Customer Feedback
Make sure to ask "Is there anything else?"
Several stories about bad feedback forms
Interviewing 5-10 customers is probably as good as interviewing hundreds
Don't assume that no complaints = customer satisfaction. They may just be putting up with it, especially if they feel no one is listening.
Don't just do surveys. Use different feedback-gathering methods. Invite open-ended feedback, in surveys or otherwise.
Don't ignore the feedback!
Focus on the service attributes most important to your clients. Don't know what's important to them? Better find out!
"What aspect of our service is most important to you? Regarding it, how are we doing?"
Lots and lots of examples of how not to
FBWA (Feedback By Walking Around)
I love how Microsoft gets all Ajaxy with feedback on every page: http://msdn2.microsoft.com/en-us/library/ms229931.aspx
Again, act on the feedback! Summary of responses, detailed responses, action
Don't forget power of the naked eye. Often problems are obvious and don't need surveys
Wednesday, September 19, 2007
Security code reviews
Hacme Casino http://www.foundstone.com/us/resources/whitepapers/hacmecasino_userguide.pdf
Foundstone CodeScout
Paros (web app security assessment) http://www.parosproxy.org/index.shtml
Don't overanalyze. (Spending two hours determining if a strcpy is vulnerable. Takes two minutes to change)
Identify code review objectives (Insider backdoors, compliance with specific regulations)
Lots of discussion of tools. I think the point is, use available analysis tools before bothering with a code review - it's easier and cheaper
http://www.securecoding.org/list
http://codesecurely.org
Usability by Inspection
Doing usability reviews?
Me either. But if you've got a product that has a UI, an easy thing to do to improve the product is to just sit down with a few people, maybe some that will actually use the product, maybe some managers, or maybe just some people that you can pull in to see what they think. That's more or less the gist of what I got from Larry Constantine's session on Usability Reviews.
Now, the first thing to realize is that a "usability review" is different than a "grouse session". I was once doing a demo for an internal tool my team was working on, and after I'd showed how the tool worked, during the Q&A period one guy spoke up to say, "Boy, does that interface ever look like it was designed by a programmer."
"Interesting," I said. "How would you improve it?"
"Oh, you know. It just doesn't look as sharp as it could."
Well, yes. Nothing ever does; but it wasn't too helpful to tell me that. So, when you do a review, you have to be specific.
But how can you be specific about a UI? A UI is just a UI, right? It either looks good or it doesn't.
Not at all! There are lots of basic principles of design that the people who make web sites for a living know about. Even if your organization really is full of web pages designed by programmers, there's no harm in teaching the programmers some basic principles of design. I have a couple of books on that subject, one by Mr. Constantine himself, which I didn't even realize until I'd gone in to the session. But the organization or team should probably lay down the fundamental precepts of design that they want to follow. The usability defects will be easier to objectively identify with that list in mind. Some examples of good design principles are: Availability, Feedback, Structure, Reuse, Tolerance, Simplicity. Check one of the books for some guides as to the specifics, but a usability defect violates one of these principles, or you could also say it is a probable cause of user delay and confusion. But it's not a usability defect if you just don't think it looks good!
So here's how you prepare for a usability review: First, organize a few use cases. You may already have them as part of your project, or you may just have to make some up. What you'll be doing is telling the users what they're trying to accomplish.
Then, get the folks together. At a minimum, you should probably have:
- A leader, to make sure everything moves along smoothly;
- A notetaker;
- A Continuity Reviewer. This is someone who is reviewing the UI specifically to make sure it is consistent with overall project guidelines, and with the other pages in the project.
- Users - people who will attempt to use the page. They can be actual customers; agile-style customers; or just people who were walking down the hall at the wrong time.
- A Designated Driver. This is someone who will perform actual mouse clicks or typing at the request of the users. This will depend on the exact situation - do you have a real application, or just some mockups? Do you have a big meeting room and a lot of users, or not? If not, the Designated Driver might as well just be the user.
- Developers/Designers. Developers and designers who worked on the page must never explain or defend design, argue with users, or promise anything. They may only find problems. Users do not count as problems.
Next, have the users go through the use case or scenario you've designed. Introduce the scenario with an overview of context and user motivation. Read one step of the scenario at a time, and ask the users what they would do next. Users take lead in proposing actions. Never guide or prompt users! Help is limited to simple description or clarification. If the user has to ask for help, you've automatically got a usability defect.
For each defect that you find, the notetaker should note:
- The feature or function that the defect is in;
- The location; which web page it is or a screenshot of the GUI
- Which design principle is being violated
- A short description of the problem
- The estimated severity of the problem. (nominal, minor, major, critical )
You should probably allow one to three hours for the review. So that's it! Get out there and say goodbye to applications that look like they were designed by programmers!
Web Application Risk Modeling
A threat is not a vulnerability. A threat is what someone might try to do to your system; a vulnerability is how they would do it successfully
What risk drivers are there?
Application overview: Documentation drill; models; dataflow
Decompose application: break it down into well-defined "chunks".
Identify threats against the security objectives
Identify vulnerabilities "Vulnerability Assessments"
A threat model helps you to define, categorize, and prioritize vulnerabilities
Make sure to fix vulnerabilities, not exploits - understand all nuances, attack potential, exploit paths
STRIDE / DREAD
Other factors:
Ease of use, mitigants, timing, visibility,
monitorability (can you watch people doing stuff?),
forensics,
access required( even for internal apps, what are the chances of a bad guy infiltrating? )
XSS: Take user-inputted data and display it back without filtering. Nuances to XSS (Reflective Script Attack, Persistent Private Vectors)
POST based attack would not show up in server logs
Tuesday, September 18, 2007
xUnit Test Patterns and Smells
Here's my history on test-driven develoment: Back in the nineties, I first read Martin Fowler's Refactoring. I thought it was a good idea, and attempted several refactorings on the code base I was working on, with good success. I think it was one of the better-coded applications to come out of that company. But I was always annoyed, because the instructions for the refactoring would always say something like, make your changes, and test. Testing is hard, man! Especially when you're testing a bit of the application that takes two minutes to get to from application launch and relying on a Direct3D driver to do the right thing.
So I added refactoring to my arsenal but didn't think too much more about it, until about five years ago, when I ran across an article on TDD in, I think, Dr. Dobbs, but it may not have been. The article mentioned some ideas about testing and mock objects, which turned out to be exactly what I needed for the project I was working on then, which was a business-level client API with a wrapper lib for calls to the server - the ideal thing for a mock. I played with it for a while, and it worked beautifully! Pretty soon I presented a proposal for moving to TDD to the team I was working with.
There were a couple of quotes that I put in my presentation (probably from the magazine article) that I really liked:
- Tests must be easy to run. If they aren't, people won't run them.
- Tests must be easy to write. If they aren't, people won't write them.
The problem is, tests are easy to skip. Comment out. Ignore. If you do that, your code isn't being tested. But the client doesn't care about that...at least in the beginning. Later on, if your code isn't being tested, bugs will start to crop up. You'll make a change in one area that you never in a million years thought would affect this bit of code over there. But it does, and you've introduced a bug. The client will sure care about that! So you really have to put the effort in to write tests.
But at the same time, you're selling the production code, not the tests. If your team is spending more time on the tests than on the code itself, your velocity is sure to suffer.
So what's the solution? Go back and look at the second quote again. Tests must be easy to write. How do we make them that way?
The first thing to notice is that your objectives for test code are probably going to be a little different than for the production code. For example, execution speed is crucial for production code. You can't have your users twiddling their thumbs while they wait for your web page to load. But for test code, not so much. Go ahead and add ten seconds worth of tests to your build; think anyone will notice? Or, add four hours worth of tests. Sounds good! Just make sure to run them overnight when no one needs to watch them.
On the other hand, is simplicity important for production code? Well...it can't hurt, of course. The smaller and cleaner you can get the code, the better. But sometimes there's nothing you can do about it; you have to add that cache for speed; or denormalize the database so you don't have to make calls across a dozen tables. But for test code? Let's say it again: Tests must be easy to write.
What else? Is correctness important for production code? Of course...but users will put up with small bugs. But correct test code is an absolute requirement. If you don't have the tests right, you'll be writing incorrect production code to satisfy the bad tests. What about flexibility? Code should be flexible, right? Not really, not test code. In fact, there will probably be enough hard-coded test values to make it hardly flexible at all.
This is getting long. I'll add more later.
Software in the large
Scrum of Scrums
Crystal
Iteration Duration: larger the team, shorter the development cycle
per week, count on a half day of retrospective (two week cycle = 1 full day retrospective)
Expectation: plan/develop/deliver.
Difficult - activity-oriented planning or component-oriented planning?
Therefore: Result-oriented planning. Focus on the features! Comes back to the Agile Manifesto: Our highest priority is to satisfy the customer.
Plan for accomplishing a valuable feature: integration, test, documentation.
A feature is a brief statement of functionality, from the user's perspective
How does one deal with architecture issues?
A feature produces a measurable result.
Iterations are steered by features, but defined by tasks
Tracking tools: PPTS, TRAC
Someone also mentions they use Sharepoint
Or just three checkboxes: working on it, untested, done done
Tools support communication, not replace it
Release Planning
Iteration review (Demo)
Present software, recognize & extract best practices, learn from failure
Measurement: Acceptance tests, planned functionality, is the product owner satisfied?
Retrospective after every iteration. Likely problem that people try to make large-scale changes
- Cross-functional or feature teams
- A large project might have tech teams; the customer of a tech team is a feature team
An ideal team is self-organized; this ensures whole features and good knowledge sharing. Managers must provide environment allowing teams to gel. This is like my ACG posts from a few months ago.
Trust
Agile development is a trouble detector. Bad news is also good news. Integration of departments (Projects are customers) Close customer relationship ensures rapid feedback.
Discussion of implementing practices a few at a time. Ping-pong implementation!
Synchronization: Face-to-face is preferred. Sync across subteams daily (Scrum of scrums). If your team is self-organizing how does that work?
Communication via wiki
Just one "Chief architect" - pulls the strings, makes technical decisions, "guiding light". Relationship of chief architect and customer?
Starting: take baby steps. Start small. Use skilled people. Develop a few features and make sure to do iteration retrospectives. Grow slowly.
Don't finalize architecture before growing team; use retrospectives. Domain teams must formulate new requirements. (But you might have to finalize to eliminate fear...or at least say it's finalized!).
Avoid hot technology. A large project has enough problems on its own without trying to train developers on something new at the same time.
Refactoring: technical excellence is doubly important. If a developer sees a needed refactoring on another team, they have to point it out to them.
Large projects may have exponentially greater test time. 10% of dev effort for integration/build. (If something is difficult, do it over and over until it's not difficult any more.)
Q: Special iterations for integration? A: no
Nor a special integration team; rather people from each team who specialize in integrating
Reviews:
Special review team. People should jump around between teams, and be on a team strictly for the purpose of reviewing the code. Everyone should do this.
Knowledge transfer (via Daily Scrum and pair programming). Scrum master ensures the process; product owner ensures business value).
Q: Agility in a distributed environment. A:
Monday, September 10, 2007
Could not load type 'Global'
I had recently upgraded an application from ASP.Net 1.1 to ASP.NET 2.0, but to keep supporting old versions of the application, I branched it off in Subversion. To make sure the old version still worked, I checked out the old code into a new folder, then I went into IIS and simply moved the location of some virtual directories to point to the old code. It all worked and forgot about it.
Until later, when I came back to make some changes to the new application, started it up and got the message:
Parser Error Message: Could not load type 'WebApplication1.Global'.Source Error: Line 1: <%@ Application Codebehind="Global.asax.cs" Inherits="'WebApplication1.Global'" %>
I couldn't make heads or tails of it, but a web search led me to Harish's post and lots of different answers, several of which I tried, but none of which worked. One suggestion was to make sure the application was set in IIS to use ASP.Net 2.0 rather than 1.1, and even to set it to 1.1, click Apply, set it to 2.0 again and click Apply, just to make sure it took. Another was to make sure the application was compiled. If there's no assembly built for the application, it won't load.
I checked the ASP version, and it was indeed set to 2.0; I said, "Duh!" to compiling, but made sure there was an assembly in the bin directory, and there was; so I was at a dead end.
But I'm sure by now you see where this is going. As I opened up IIS to check the ASP.Net version configuration again, I happened to glance down at the local path for the virtual directory on that tab. And what did I see? The directory was still pointing at the path to the branched directory; a perfectly legal application, but one that was built using ASP.Net 1.1, and also had been cleaned sometime in the not-too-distant past. So the version I had configured in IIS was neither compiled, nor a 2.0 application! No wonder the error came up.
So I have an additional solution for this problem. Check your virtual directory location and make sure it's pointing to the application you're expecting it to be.
Thursday, September 06, 2007
Pair Programming with VNC
Monday, August 27, 2007
School Daze
Tuesday, August 21, 2007
Deconstructing the Wiki Decision
That said, how likely is it that students are interested, under their own power, in editing a wiki? Based on my experience that only about 5% of readers tend to be contributors, I think it might be difficult - but of course, the percentage in an English class might be higher. Students around here are asked sometimes to edit Wikipedia or Bloomingedia as a class assignment; for example this article:
http://www.bloomingpedia.org/wiki/Amused_Clothing
was obviously written by a local teen. But if you look at the contributor's history, he copied in the bulk of the article on March 30th, came back and fiddled with it a few days later, and then never came back again.
Now, Mr. Long isn't having his kids put their essays on the wiki - at least not yet! But if, or when, it occurs, I wonder whether an English class discussion wiki would really work on its own terms without constant prompting by teachers. I suspect it could, if it is linked to the real world somehow. I'll be following the experiment with interest.
Wednesday, August 15, 2007
Bloomington not yet a'Twitter
While James was posting his updates, I tried to follow along with his numbers on a Google spreadsheet, with only a fair amount of success. (Of course, my job was easy since all I did was read the comment threads. James had to try to interpret everything and post and try to keep up with details on the numbers - all in real time.) My goal was fairly self-centered: I wanted to understand exactly what they were voting on and why. But certainly if what I was doing was useful at all I wanted to share it - why keep it private? (A former boss asked me that once. Why did I blog about my trip instead of putting it in an email and sending it to the six or seven people in my group? All I could do was stare at him blankly.) All in all, I'd say that my, and probably a lot of other people's, information stream had gotten a lot wider this week.
Thinking along many of the same lines, only way more articulately than I could ever be, Kevin Makice wrote an piece on the future of local social networking. Kevin wants everyone to center around Twitter, which I doubt will happen. The Herald-Times has taken a real leadership role in this process, and they of course have a vested interest in bringing people to their site instead. Councilmember Sophia Travis pointed out that it was way too tough for her to actively participate in the discussion as well as listen to the issues, although she did manage a couple of notes.
So where do we go from here? Here are a few things I notice:
- It took a professional, not a blogger, to (a) generate interest and (b) pull off the budget updates with the right amount of elan to keep everyone interested. Is this a requirement? I'd say no, but the fact is that I wasn't about to take several days off work to go down there and watch. It's a lot easier to do it if someone will pay you.
- With the exceptions of Councilmembers Travis and Marty Hawk (who posts to the HT occasionally) there are few enough politicians in the general conversation, to expect that there will be many in the live conversation (by which I mean Twitter, or the running comment thread). It would be nice if this changed.
- I had to ask early on in the process for copies of the spreadsheets the council was using. Apparently the auditor was running around with them on a thumb drive, handing out copies to whoever needed them. It would have been nice to just stick them on a web page at the beginning.
- I want a budget expert available to answer questions from the public. I probably had a dozen questions over the three days - granted, I always have questions, it's because I don't know anything - but many of them James couldn't answer, and probably many he could have but didn't because he didn't have time. Wouldn't it have been cool if the auditor's office could have somebody sit and monitor the thread and explain stuff?
- Let's not wait for next year's budget to do this again. Send the junior copy editor to update us on the Redevelopment Commission meeting. Let's get a volunteer blogger to liveblog the Planning Commission. Let's keep the government exposed!
- Budget hearings are a really moronic way of doing things. A bunch of exhausted people sitting in a room voting yea or nay at random on a couple of grand so they can get it over with and get some lunch? Tell you what, next time let's get all the line items out on a nice wiki page and hash it out that way. I realize I'm text-centered and maybe others prefer the face-to-face, but then how about over NetMeeting or something?
- Now, I'm not trying to grouse and say that things should have been done differently. Or to be more precise, of course they should be done differently, but we never know precisely how until afterwards. This has been a great learning week for me, and I hope, for everyone else as well.
Sorry, Kevin, I didn't get that Bloomingpedia article on the budget written; the hazards of citizen journalism :) But maybe now we all see a little bit more of the possibilities that are opening up before our eyes. Hey, follow me on Twitter!
Monday, August 06, 2007
Dare Obasanjo on Open Social Networks
I learned via TechMeme, though, that Jeff Pulver is leaving LinkedIn for Facebook. I think it's a mistake, Jeff. LinkedIn is specialized; it exists for business contacts. It will probably be around in a couple of years, linking up business contacts. Facebook will probably be gone as people move on to the Next Big Thing.
To sum it up, it appears to me that the real evolution of social networking is going to be LinkedIn for business contacts; Flickr for pictures, LibraryThing for books, and then maybe a few small sites like Facebook and MySpace that aggregate all this data into a coherent whole for people who aren't interested in creating their own websites that aggregate all this data, or are nervous about being outside of the walled garden. But Facebook ain't the future. Don't expect it to be.
Saturday, August 04, 2007
Your code is suboptimal!
Friday, August 03, 2007
Out of the Theater, Into the Courtroom
Wednesday, July 25, 2007
Javadoc Clutter
So where do the diagrams come from? As Alfred mentions, you can use class designers like the one in Visual Studio, but my feeling is that that is only a starting point. There are so many different diagrams you can make: dataflow, inheritance, etc., but you have to keep in mind that the point of any diagram is to help the reader grok the system. What I like to do is keep a documentation wiki around, and generate some diagrams that can be added as pictures, and as a starting point for some user-defined text to help explain them.
But when you do that, eventually you're going to want hyperlinks in the text that lead back to the class, and its description, and its methods. And this is where Javadoc comes in. In the build, throw in a step that generates HTML pages from the Javadocs, and make them available to the users of the project wiki. I think this gives you the nicest combination of high-level overviews and class-level references, both of which are essential to a well-managed project.
Monday, July 23, 2007
The 20 Dumbest Words in Software Development
What I have written here is short, and therefore oversimplifies the many issues. But the full range of agile practices can answer most objections, in my experience.