There have been some issues filed recently about exceptions being thrown when running NUnit tests under the VS test window. In many cases, they result in trying to install two different versions of the NUnit Engine into the same project directory. I'll try to explain how the problem arises and what to do about it.
I'm in the process of moving many of my old posts to a new website - this one - and that involves thinking about some things I have not thought of in years.
It's the Tests was my blog about Agile software development, Extreme Programming and Test Driven Development for many years. I have gone through the posts and included those that I feel may still be of interest under the "It's the Tests" tag on this site. The entries have been edited lightly, to fix broken links. I have kept some comments and removed others tht no longer seem relevant today.
A post on Twitter claimed that Agile won't work for a heterogeneous team because it's practices were developed by a "bunch of white dudes." While that isn't something I want to believe, the comments indicated that many folks did believe it. When a large body of opinion contradicts my assumptions, I like to pause for thought.
Some folks have expressed surprise at my release of NUnit 2.6.5. Their surprise is no surprise, given that the NUnit framework is now at version 3.10.1!
I'm a big fan of microtests - both the term and the thing itself. My friend Hill coined the term quite a while back and I felt it completely solved the problem of ambiguity we agile folks were having when we talked about unit tests in front of people who understood the term in the way it was used 30 or more years ago.
A while back I began to have some concern about the future of NUnit. I was entering my 70s and I knew I wanted to spend more time on other things. NUnit had been very much my project for a few years and I didn't want it to die when I was no longer maintaining it.
UPDATE: I'm leaving the post here but the action described has been reversed and the project continues to live at https://github.com/charliepoole/nunit-summary
My blog has been offline for a long time, as you can see. The last prior post was in 2009!
In a recent online discussion, one of our users talked about needing to re-run the NUnit console runner, executing just the failed tests from the previous run. This isn't a feature in NUnit but it could be useful to some people. So... can we do this by creating an Engine Extension? Let's give it a try!
Let's say you have an array of ints representing years, all of which should be leap years.
NUnit 2.5 has so many new features (see the release notes) that I thought I'd try to come up with my top-ten favorites. It was hard to get down to ten, but here's what I came up with...
The latest code for NUnit 2.5 includes seven generated files, including the Assert class and most of the classes that allow you to write constraint expressions using the NUnit fluent syntax. Some people have asked if generating these files is worth the effort, since the code created is very simple anyway.
Ben Hall has posted a nice summary of the Parameterized Test Features in NUnit 2.5 Alpha-3. Of course, being from the UK and all, he calls them "Parameterised" Tests.
Let's say we are testing a piece of code, using arguments that should cause an exception to be thrown. We want the test to ensure that an exception was thrown, that it was the expected Type of exception and - possibly - that the properties of the exception are what they should be.
One recent addition to NUnit 2.5 is the ability to define generic test fixtures, allowing the same fixture to be reused for multiple types that implement the same interface or even just having common method signatures. For example, the following code tests multiple implementations of IList.
Thanks to a contribution from Microsoft, kindly arranged by Stephen Walther, I now have a copy of Visual Studio 2008 - and NUnit has a VS2008 build! It's available in CVS and will be part of the Alpha-3 release in a week or two. NUnit operates pretty much on a shoestring these days, now that I'm semi-retired, and contributions like this help a lot.
It has been possible to write parameterized tests for NUnit for some time, using Andreas Schlapsi's RowTest Extension, which recognizes the RowTest and Row attributes familiar to MbUnit users.
With a European trip about to start, I decided to release a second Alpha so that the new stuff would get some visibility. I won't be doing another release till late June, so please give this one a try.
Scott White has a blog about NUnit Best Practices. The approach may require adjustment on more complex projects, but it's a very simple recipe for those starting out with NUnit.
Let me start right out by saying: I know some of you won't find this to be simpler - but it is! If you can't bring yourself to install NAnt or work from the command line, then this isn't for you. But if you can get past the initial hump - or if you're already past it - then this is for you.
A recent bug pointed out that addins are not recognized when running tests under the console runner. This is due to a missing entry in the nunit-console.exe.config file, which you can easily fix yourself. Follow these steps to have your addins recognized when using the console runner:
A few folks are confused by the various release numbers being announced or discussed all at one time, so I thought I'd clarify:
As I've worked on NUnit 2.4 and its follow-ups, I have begun to feel that it's time for a much more significant update to NUnit, a rethinking of what it's all about and how it should work. Of course, at the same time, I don't want to lose all the work that has gone into NUnit up to now. Puzzling over this, and helped by a group of dedicated NUnit users and contributors, I think I now have a direction to go.
While NUnit 2.4 was in development, I continued to maintain the NUnit 2.2 series as a stable, bug-fix-only branch. NUnit 2.2.10 was the last in that series.
I haven't done much blogging for a while due - in part - to the enormous amount of Spam my blog was attracting. But I just updated the software and installed a spam filter. I plan to start posting again, and re-open the blog to comments. Hopefully, the spam won't overwhelm me this time.
When running under .NET 2.0, NUnit is rather slow at loading tests these days. Although many folks only noticed the slowdown with the release of NUnit 2.4, loading a large set of tests with NUnit 2.2 also takes more time - about twice as much - under .NET 2.0 as .NET 1.1.
NUnit is currently built under five different runtimes on Windows - plus two on Linux. Each build is tested under the same runtime on which it was built and on a number of the other runtimes. That's a lot of TestResult files.
NUnit 2.4.2 is out and available here. It includes additions to the new constraint-based Assert syntax and new attributes to facilitate testing under various culture settings. The gui has been enhanced to save the visual state of the tree, including the currently selected test, and to allow loading just the fixture you are working on, which saves a great deal of time on reloads.
I just released NUnit 2.4. It fixes a few issues that slipped into the 2.4 release an makes some minor improvements. I wanted to get this release out because I'll be travelling for a few months and won't be in a position to do further releases until July or August.
Although NUnit 2.4 is almost out the door, there were a few outstanding bug fixes to the NUnit 2.2 series hanging around, so I decided to issue a final 2.2 release. Like all the releases since NUnit 2.2.4, this is primarily a bug fix release.
NUnit 2.4 RC2 is out now, correcting a naming conflict with several mock object frameworks that was present in RC1. You can download it at NUnit.org. For a full list of the extensive new features in NUnit 2.4, check out the Release Notes.
The release process for NUnit 2.4 has taken long enough that a number of bug fixes accumulated on the NUnit 2.2 codebase. I've just released NUnit 2.2.9, which addreses a number of bugs.
As pointed out by Sara Lee McLindon, the SampleFixtureExtension example in the NUnit 2.4 Beta 2 release doesn't actually work. You can correct it by adding the following method to the SampleFixtureExtensionBuilder class.
NUnit has lots of tests of its own - unit tests that is. It even has some fairly high-level tests that don't fit well into the normal unit-testing paradigm. But, up to now, the only acceptance tests were manual - a list of things I do before uploading a new release. Since they are manual, they don't get run all that often and surprises happen.
In an earlier post, I presented some ideas about syntax for expressing assertions in tests. I was doing this as a part of the development of NUnitLite, with the idea of eventually putting some of the same concepts back into NUnit.
NUnit has finally gotten rid of it's Visual Studio Install projects in favor of WiX
Sometimes you expect an exception to be thrown by a method. So, of course, you want a test for that. NUnit provides the ExpectedExceptionAttribute for that purpose. It has a bit of history...
I've been using the development of NUnitLite (coming soon!) as an excuse to try out alternatives to the standard NUnit syntax for Asserts.
In an earlier article I wrote about my concern with OpenDomain.org. They had acquired the nunit.net domain and were offering me the right to use it in exchange for a link. I felt uncomfortable about this, particularly after I learned about their dispute over the WordPress.com domain.
At OSCON, I spoke about NUnit for Cross-Platform development. Here are some screenshots from that talk, showing NUnit running under Mono 1.1.13 on Ubuntu Linux. (Click to enlarge)
People want what they want. If NUnit doesn't have a feature that they need, they ask for it. Of course, we can't implement all of them, so some people want to be able to create there own test runner using the NUnit assemblies.
This week at OSCON 2006 I finally learned some Ruby. I've been meaning to do this and learning by listening to Dave Thomas and Mike Clark seemed like it would be much more fun than simply reading a book. Turns out I was right.
A post on the TDD list asks about running NUnit tests using the Start button in Visual C# Express. In Visual Studio 2003 and 2005, you can set up a project's Debug properties so that an arbitrary external program is executed. By selecting NUnit as that program and providing the correct command-line options, you can cause your tests to be run.
In my last post, about the NUnit team's plans for trying out the Microsoft CodePlex site, I introduced the NUNitLite project. One thing I mentioned was the possibility of its being released under the GPL license. This seems to have gotten more reaction than anything else in the post. I find that a bit disappointing, because I thought there were some other cool things in it. :-)
Be that as it may, I'll try to explain here why my next software project might use GPL and what kind of considerations I'm looking at in making a final choice.
A reader asks "What is this about?" pointing to the new Microsoft CodePlex site, which mentions NUnit.
NUnit originally identified tests in the time-honored way that is still used by most xUnit frameworks. Test classes inherited from the framework's TestCase class. Individual test case methods were identified by their naming pattern.
I'm afraid I'm pretty erratic about announcing new NUnit releases on the blog. NUnit 2.2.8 was released on April 21, 2006. It has a number of bug fixes to the original 2.2.4 feature base.
There are some traps involved in running tests from code that are not immediately obvious. This was brought forcibly to my attention when I recently worked on a failing test that someone else had written.
The test in question is internal to NUnit. It attempts to run a single fixture out of a particular assembly, and then verifies that the results are as expected. I'll cover some of the traps encountered in the order that I resolved them.
Last week I attended the first XP Day France, where I had been invited to give the keynote. We don't see many attendees from France at the European XP events, which is too bad, because the quality of the presentations was impressive. I'll be blogging separately about some of the things I learned.
We could all benefit by hearing more from these folks and I think they would also benefit from being part of the larger events. I hope some of the folks I heard speak here in Paris will turn up for XP2006. If language is an issue for some, I'd be glad to help.
A Chinese programmer (anonymous until he tells me otherwise) recently wrote to ask me for a copy of NUnit 2.2.7. For some reason, it appears that Sourceforge is not accessible in China. This seems odd to me, since it's hardly a political site, but the ways of bureaucracy in any country can be hard to fathom.
NUnit 2.2.7 has just been released. It fixes some bugs that were filed against the 2.2.6 release, although some of them had been in the code for a while. You can read about what's fixed in this latest release, as well as earlier releases, in the Release Notes.
In my earlier post, The Death Of NUnit - Will it be Ice?, I adopted the notion - promoted in some mailing list posts - that NUnit might be dead or dying. I identified two big worries: death by ice and death by fire. Ice stood for frozen inactivity on the part of the project itself - failure to move ahead. Fire symbolized competitive forces, which might make NUnit obsolete.
I've been occupied with other things and not blogging lately. Meanwhile - in fact this is one of the things I've been occupied with - NUnit 2.2.4 was released. People tried it and found a few bugs - NUnit 2.2.5 corrects them.
The NUnit 2.2.3 release contains two versions of each download: one built with .Net 1.1, one with .Net 2.0. So, what's the deal? Do we need separate versions? And which version do you need? I'll try to explain...
We forget sometimes how easy it is to write tests - even without a framework. I was sitting in a cafe, working on my new laptop. It's an Acer Ferrari 4000 that I'm very happy with, but I haven't completely set it up yet.
A recent post on the NUnit Open Discussion Forum on Sourceforge asked “Is NUnit Dead?” I have to admit: the question irked me a bit. But reflecting on it has led me to a few conclusions that I’ll share here.
The opendomain.org site seems to have acquired nunit.net, a domain we thought about using but didn't. Now I'm sorry, since they could use it for pretty much any purpose.
There's a discussion going on right now on the Yahoo Test-Driven Development list regarding the use of a single instance of the user TestFixture class in NUnit 2.x, as compared to multiple instances in NUnit 1.x and in JUnit before it. A related issue, discussed on the same list a while back, is the question of the lifetime of that single instance. Here's my take on how the issues relate and how NUnit deals with them.
Lots of people write to ask about the use of configuration files with NUnit. Usually, they have put some setting in a config file - somewhere - and are wondering why NUnit doesn't seem able to find it. In this post, I'll try to summarize the main issues around NUnit's use of configuration files. It really isn't all that complicated, once you understand a few things...
In a discussion about unit-test suites that take too long to run, Ron Jeffries writes
Splitting the tests into slow and fast is a tradeoff, and it's an easy one. But is it ideal? I think not. I think a better approach might be to split the tests into "likely to provide interesting information" and "unlikely to do so". Then make the ones that are likely, also fast.
This struck me as an interesting point. If the value of a test is seen as the amount of information it is likely to provide and the cost is - at least in part - the time to run it, then the problem of which tests to keep in that "too slow" test run is quite similar to the value versus cost balancing problem we face in XP when we schedule stories into an iteration.