LINQ with DataReaders

Over the past couple of days, one of the MVP lists has had some activity about LINQ (which is pretty common these days), and one of the threads meandered into using LINQ on DataReaders. The problem is that LINQ works on IEnumerable sources, but DataReaders don't implement that. Fortunately, it's easy to do, and several people out there have come up with various solutions, including Jim Wooley in this article of his. The basic gyst is that you can you create an extension method on IDataReader that serves as an Enumerator.

However, Jim encountered two problems. He is returning the data one row at a time (which is the point) but as an array of object. So first, he's worried about performance. This is a valid concern considering the boxing/unboxing implications for all value type columns as well as the fact that a LINQ statement might try to grab those values several times (for sorting, as an example). Secondly, because all you have is an array of object that represents a row, you reference each column of the source as "row[0]" and "row[1]", etc. - basically, you don't get to use nice column names in the SQL.

The way to fix both these problems is to have an object that represents the row, complete with specific and type-safe fields (properties if you want to get technical and PC). Then, you need a way to convert each DataReader row to an instance of the row class. Expanding on Jim's example, here's what I came up with:

C#
public interface IReaderRow
{
    IDataReader Reader { get; set; }
    object GetRowData();
}

public class TimeZoneRowReader : IReaderRow
{
    public IDataReader Reader { get; set; }

    public object GetRowData()
    {
        return new TimeZoneRow() 
        {
            ID = Reader.GetByte(0),
            Name = Reader.GetString(1),
            Description = Reader.GetString(2)
        };
    }
}

public class TimeZoneRow
{
    public int ID { get; set; }
    public string Name { get; set; }
    public string Description { get; set; }
}

public static class DataReaderExtension
{
    public static IEnumerable<T> DataReaderEnumerator<T, TReader>(this IDataReader source) where TReader : IReaderRow, new()
    {
        if (source == null)
            throw new ArgumentNullException("source");

        IReaderRow rowReader = new TReader() { Reader = source };

        while (source.Read())
        {
            yield return (T)rowReader.GetRowData();
        }
    }
}

As you can see, the LINQ statement is now typed and has intuitive and descriptive names for the columns. The downside, of course, is that you need a class representing each type of possible result, and a way to translate the DataReader data to that row instance. You can combine the row class with the specific row reader class, but I simply chose to keep them separate. Strangely enough, this starts to feel like LINQ to SQL or LINQ to Entities at this point, but it's a way to solve the two other problems I mentioned earlier. In the end, the LINQ will now look something like this:

C#
listBox1.Items.Clear();

using (SqlConnection cn = new SqlConnection(connectionString))
{
    using (SqlCommand cmd = new SqlCommand("Select * from TimeZone"))
    {
        cmd.CommandType = CommandType.Text;
        cmd.Connection = cn;
        cn.Open();
        using (SqlDataReader dr = cmd.ExecuteReader())
        {
            var queryResults = from timeZone 
                in dr.DataReaderEnumerator<TimeZoneRow, TimeZoneRowReader>)
                where timeZone.ID > 0
                orderby timeZone.Name descending
                select timeZone;

            listBox1.DisplayMember = "Description";
            listBox1.DataSource = queryResults.ToList<TimeZoneRow>();
        }
    }
}

If you search around, you'll see quite a few other solutions (some of them in community projects) for using DataReaders as LINQ sources, although they all seem to have some of these aspects in common. The really interesting stuff is when you start to think along the lines of using combinations of IL Emit and anonymous classes to create all these row-related classes automatically (the schema is available on the data reader, after all). However, without getting too tricky, it's tough to find a way for C# or VB to interpret the results in design time so you can still maintain the intellisense and column names without resorting to something like code generation.

More P/Invoke Help

One of the first things I really hammered with .NET 1.0 back in late 2000 was P/Invoke. I was used to Java, but IMO, Java always had a fatal flaw - language design hubris. Java is perfect, and C is garbage, therefore, everything should be converted to Java, or so it seemed anyway. That philosophy trickled down to how Java used existing C code - JNI (at the time) wasn't a way for Java to call C code as much as it was a way for *YOU* to write a "clean" and extensive wrapper worthy of being called by Java so that Java wouldn't have to dirty its hands with the "bad" C code. At the time, I was extremely skeptical about .NET, but one thing I did like is that it readily admitted that the enitre world was not in .NET and there was plenty of "legacy" code written in C that was.. *gasp* ... usable and useful! Furthermore, it cooperated with said C code and even had a very simple (relatively speaking) way of calling it. But a lot of people weren't used to P/Invoke, although it was similar in concept to using Declare statements in VB (just more powerful). I was really taken by how comprehensive P/Invoke was, but people who weren't used to looking at C code had a hard time translating calls. I remember writing a little tool called P/I-Spy that would take C function prototypes and convert them to P/Invoke, or do the same work via a step-by-step wizard that allowed you to describe the call. The problem is that I wanted to create a comprehensive database of calls for the Win API, and I just didn't have the time, so I was happy when someone else took up the mantle and created http://pinvoke.net. At that point, I let the tool slide completely and have never touched it since.

However, Microsoft just launched a tool of their own. You can find it here: http://blogs.msdn.com/vbteam/archive/2008/03/14/making-pinvoke-easy.aspx, and it has some really cool features, among them, the ability to convert volume code.

Saying YAGNI to YAGNI?

I was reading a very interesting post this morning: http://codebetter.com/blogs/james.kovacs/archive/2008/03/13/using-yagni-responsibly.aspx

and I want to start by saying that I whole-heartedly agree with James at the heart of the matter. YAGNI, when intelligently applied, is not just an excuse to do stupid simplistic things. But that got me thinking (again) that YAGNI itself is probably an inadequte piece-of-jargon-turned-tool that encourages some people to use it "unintelligently". My motto is that if something isn't providing consistently good results, then there's probably something wrong with it, so here's the gist of my thoughts:

I think the problem with YAGNI is that as a concept, it works more like a binary switch. It encourages people to think in Yes or No options and concentrate on a single vector of a multi-dimentional problem. By "intelligently" applying YAGNI, the focus should really be on providing the most effective and efficient (i.e. simplist) solution that can solve the problem given all the goals that must be met. This is not a "yes I need it", or "no I don't" question - it's a question that begs us to consider a range of alternatives and evaluate them based on (not just complexity) but R.O.I, maintainability, and likelyhood of change and stability, among other factors. At this point, some agilists might argue that we're trying to do too much design upfront by taking all this into account, but most of these factors can be very quickly assessed in a matter of minutes on a "good enough" basis, which is really in keeping with the spirit of Agile, IMO. YAGNI is just a simple(istic) way of terming it, but perhaps it's too simplistic and encourages us to evaluate only one vector (complexity) of the problem that really has other dimensions that need attention. I honestly think it hurts the process to give jargon undo weight. A person who uses YAGNI intelligently is actually taking those other factors into account, so perhaps YAGNI as a concept is just putting too much focus in one area for those who are more prone to simply use the jargon at face value as opposed to "intelligently" applying a tool, which is starting to make me wonder if it's actually a good tool to begin with.

How many factories does it take to...

So a friend and I were talking about specific CABs, and containers, and DI, and MVP, and AOP/Cross-cutting injection, when I stopped and had to laugh. It reminded me of this post: http://discuss.joelonsoftware.com/default.asp?joel.3.219431.12

At this point, some of you may be saying, "But Rob... OK, so a lot of these 'frameworks' are getting out of hand, but haven't you posted previously about how most modern software is completely unmaintainable without frameworks?" Yep, guilty as charged. However, to quote Einstein: "Make everything as simple as possible, but no simpler." And this gets to the crux of this year's push for me - how to convey that there are two equally horrible extremes we need to avoid: being so simple that it's too simplistic to work correctly, consistantly, predictably and be maintainable, and being so complicated that it borders ridiculousness, just because someone slapped a "Best Practice" sticker on it. Unfortunately, the tool needed to make those distinctions is not found in a book, blog, or manual - it's found between your ears, where it for some reason appears to be more difficult to locate. And before anyone gets overly defensive, the point of the post is not to say that the things I mentioned up top are bad - in fact, they are far from it. Just saying that before you download and start figuring out how to use someone's ShineyNewFramework(tm) v2.5, make sure you know why you are using it, and don't just do it because someone's blog said you ought to.

Exposure - Expenses to Assets

Are your developers expenses or assets? Technically, the accounting department and the IRS have an answer for that, but here's what I'm talking about: An expense is something that you throw money at and it depreciates in value or is consumed until it's done. A good example would be papers and staples. You shell out money for paper and staples, and they eventually run out, but what are you going to do? The company needs paper and staples so you have to pony up the expense. An asset, on the other hand, is something for which you pay now, and expect future benefits or returns. In other words, think investment. It's not just tossing money out the window, it's (eventually) getting something in return for the initial cost.

So with our simplistic analogy, investments are only good if they grow and provide a bigger future return. You have the choice of viewing your developers as a necessary expense, or viewing them as something that with a little investment, will provide greater future benefits for the organization. But in order for that to happen, developers need to grow in skills. When hiring, the best developers I've gotten are the really passionate ones who eat, drink, and dream code. They spend a ton of their own time researching and collaborating and growing their skills. But realistically, not all developers are like that. It works out mostly if you have single guys and gals who have that level of interest in coding and architecture and probably little else to get in the way barring the prospect of a nightly raid in Warcraft. But for the rest of society, you have parents and people with other adult responsibilities that demand inordinate amounts of time and energy outside working hours.

And that gets us to the crux of today's post - exposure. You don't necessarily have to send developers to really expensive training in order for them to grow. The key factor is exposure. Exposure to techniques, technologies, practices, bugs, workarounds, etc. The more exposure the better. For the devs who do little else in their free time, they are getting that exposure through books, articles, online forums, collaboration projects, and many other avenues. But for the devs who can't really afford to do all that (or as much), the only exposure they really get is what's in front of them 9 to 5 - which is to say, little or nothing at all that will help them grow beyond what they already do. And here's where I encourage managers to allow their developers to surf the web, be active in tech forums, contribute to a collaborative project, blog, and otherwise get the type of outside exposure they can only get online *on company time*. Sure, you have to limit it and make sure it's appropriate, but even a half hour a day can help to vastly improve their exposure to new tips and tricks, technological features, practices, and feedback they would have otherwise never gotten. Try it and see if this doesn't provide better benefits than forcing them to go heads-down in the project code 8+ hours a day.

Real Technology Heroes

OK, I'm probably *really* late to the ballgame on this one, but this site just made me laugh in that geek-humor way that has all the "normal" people around raising an eyebrow because they have no idea what's going on.

http://realtechnologyheroes.com/

So here's to you, Real Technology Hero. You know who you are!

Security and Permissions

Nothing really new in this post, but I just ran into some specs defining what needed to be locked down for a project, and it reminded me of a recurring theme that I though would be a good topic to blog about. That usually happens when you find yourself running into the same conversation multiple times.

The basics of an authorization system are pretty simple - you have users and you define what actions users can take on certain "objects". In Windows, for example, those objects happen to be files, folders, registry keys, computers, etc. In any given system, there might be many objects and actions, so for the sake of convenience, you group everything into Roles or Groups. Instead of creating an ACL (access control list - a list of actions a user can take on specified objects) for an individual user, you create ACLs for Roles. It's easier to add users to three roles (with preassigned permissions) than it is to assign 150 permissions to each user.

In some systems, the ACLs are somewhat implicit, and the code simply looks at roles. Each chunk of code looks for certain roles assigned to the user and decides if certain actions are appropriate. While this is certainly easier to implement, it lacks some flexibility and maintainability since it's a bit more hard-coded than actually having a data-driven and explicit ACL. Instead of being able to move permissions around, the most you can do at runtime is move users between roles.

Regardless of whether the ACLs are more implicit or more explicit, the meat of this post is more about "what" to lock down. I'm not sure if this is something taught in a some extremely popular systems design book or something, but it seems like BAs and users naturally drift towards wanting to lock down UI elements. The specs usually read "X users can't see/interact with Y button and Z fields on Screen A". I'm not exactly sure why that irritates me so much, but what you really want in most cases is to restrict actions to more abstract entities, rather than restrict actions to specific UI elements. If you say that X users can't Edit the contact information for Customers, or that Y users can't Deactivate Customers, or that Z users can't Create Customers, then no matter where those interactions occur on a physical UI, the same rules will apply. That's much better than defining permissions on a field-by-field, button-by-button basis. What if there are multiple ways to deactivate a user? For example, a button on Screen A, a maintenance panel on Screen B, and a Menu entry?

Of course, users will still find legitimate reasons for locking down certain UI elements - screens, reports, buttons, etc., but in my opinion, that should always be the exception rather than the rule.

OOP and Interviews

Just got back from vacation and read a blog post by Eric with some concerns about developer knowledge on OOP. It reminded me of a similar conversation I've been having on and off with friends for years, so it seemed like an appropriate topic to write a quick post. After that, I'll also toss in a quick synopsis about interviews.

Eric claims that far too many devs don't really know OOP. To some degree he's absolutely right. I see the same thing when I conduct interviews. There's an interesting dynamic when it comes to OOP, and it depends on how you approach the topic. If you ask a lot of people about OOP, you'll most likely get one of two answers - either they feel it's great (whether they really know much about it or not, they feel the industry [read: you the interviewer] thinks it's important, ergo, it's a Good Thing (tm)) or they feel it's a complete waste of time and far too complicated. Since apparently a good chunk of developers can't articulate the pillars of OOP, some people have even written entire papers asking the question "Was OOP a Failure?"

I'm afraid the RIP engraving on the OOP tombstone is entirely premature, but here's what I've discovered: Just about everyone LOVES consuming OO code, but a good chunk of developers don't like writing OO code (or simply don't know how). OO libraries are easy to consume. It's simple to understand MyMainForm.Text, AcceptButton.Enabled, and MyData.Fields["RowID"].Value. Additionally, the data and behaviors are easily discoverable. That is to say, you start with some object, and in most IDEs with most OO-languages, you click "." and you get all the object's fields, properties, and methods. There's a good reason for all this goodness - one of the primary goals of OOP is code organization. This is much easier than having tons of variables strewn in dozens of files, with names like gdwmaxdaysinqueue, CUST1_TEMP_DBID, or even X$. While some devs might even rebel against writing OO code themselves, you'd be hard pressed to find (for example) a .NET dev who would give up the .NET framework classes for winforms in favor of something flat like the Win32 User API.

The problem is that getting all the organization isn't free. Just like living in a house is infinitely easier than building a house, consuming OO code is much simpler than actually writing it. At the same time, most newer languages are making this much easier. Having said that, I also strongly believe that a lot of the "complexity" is in some respects purely perceptual. I often say that when faced with a problem, most programmers simply bang away and try samples until they get something to work. After that point, whatever happened to work becomes "the way to do it". Problem solved, why go back and reinvent the wheel? However, writing good OO code requires a fair bit of upfront thinking rather than the "discover a working solution as I go" approach. A lot of developers, particularly self-taught devs, get passed this experience and don't like to go back and discover a different way to write code to solve the problems they're already solving now, albeit somewhat disorganized. And before I get beat up for saying that, it isn't to say that only OOP code is well organized and everything else is not. In reality, I find that the most resistance to OOP comes from people who have programmed for years without it. Some of the newer bunch simply don't know much about it, but aren't really against it.

To prove the case to myself, I taught several semesters of OOP at USF. The students (most of whom were taking the class as part of a business or accounting curriculum rather than CS, some with no prior programming experience) had absolutely no difficulty at all getting the concepts with a minimal amount of proper guidance. At the end of the 7 week course, they simply couldn't fathom writing code any other way. But unlike the trial-and-error method of most self-learning, not everyone has the patience to get the guidance necessary to get into OOP, even if it comes from a book and not a teacher.

And without further ado or even a good segue, I'll dive into the interview stuff.

Interviews really serve one purpose - to see if the person in front of you can do the job. This has as much to do with technical knowledge as it does with a "fit" in the team. What an interview shouldn't be is a game of trivial pursuit. If your candidate will be writing a lot of OO code, then of course it makes sense for them to have a very good understand of OOP and be able to intelligently articulate the pillars of OOP. In today's environments, OOP is often an extremely relevant topic, which candidates really should be (at the very least) brushing up on.

However, should a candidate be able to explain how to correctly write a custom WCF transport off the top of his or her head? In 99.5% of the cases, I'd wager the day-to-day tasks the dev will be performing has nothing to do with that. Some interviewers use the interview as a vehicle for conveying Alpha developer dominance (i.e. showing off who is more intelligent or technically savvy and making sure the candidate knows he is at the bottom of the totem pole should he get the job). The questions are about things the candidate won't be experiencing on the job and any answer wouldn't necessarily qualify or disqualify the candidate from the real duties they would have to perform. That's just bad form, and if I caught anyone on my team doing that, they would really be in for it.

The same goes for that incredibly stupid bucket of trivia puzzle questions concerning light bulbs, coins, minimum distances, and trains. A lot of people claim they use these to gauge "problem solving skills". I claim BS. In fact, most of the people asking these ridiculous questions didn't know the answers themselves until they were told or found them in a trivia puzzle interview questions book. If you are asking these questions, chances are that you (a) don't know relevant questions to ask and are using these as fillers, or (b) are just trying to show off. Knock it off; you're not impressing anyone (and if you're impressing yourself, seek professional help as soon as possible). A lot of candidates are nervous enough as it is. Ask questions relevant to what they will actually be doing (and no, it's not trying to figure out how to find defective pills in the minimum number of weighings). If the open position was about Mensa Puzzle Solving Punditry, it would be called "Mensa Puzzle Solving Pundit" instead of "Sr. .NET Developer".

Then what should you be asking? Well, I'm not going to be presumptuous enough to tell you what is the best way for you to interview. In the end, that's really for you to decide. My approach is simply this: talk to the candidate. Anyone who can do the job can hold a very engaging conversation about the relevant topics. I avoid the trivial pursuit approach at all costs. Typically what I do is start vague and steer the conversation into more and more specifics as it goes. For example, find a previous project on the resume that claims to have incorporated some of the relevant techniques or technologies. I start by simply asking the candidate to talk about that project.

I look for a few tell-tale warning signs. For example, I don't want to hear "we did X" and "we implemented Y". I'm not hiring your entire team. I'm trying to hire you. I want to hear what you did specifically, not what your team did. I want to know what you know, not what your team knows. Sometimes people who (involuntarily) get off a project hide behind the accomplishments of others and don't understand the relevant topics at all. If I hear too many "we"s, I immediately start asking specifically about what the candidate himself did and understands about the project.

Also, if (as a candidate or interviewer) you don't understand a topic, don't talk about it. Nothing bothers me more than BS during an interview. I'm not looking for perfect knowledge. I'm looking to see if someone is qualified enough to do the job. Don't make things up. Don't throw around acronyms or buzz words you don't understand. If you get caught (and particularly pray I'm not the one who catches you ;-)) you will look 10 times dumber than you probably are. It's OK to say you don't know, or don't know off the top of your head. I will then try to assess how you would solve such a problem involving that topic, and that's more important to me than knowing every answer from memory. If you can figure out something after reading a help doc for 15 minutes, that says a lot about your ability to perform and produce given something you didn't already know from the start. If you are an interviewer and you do this, and the candidate catches you, you will also look 10 times dumber than you probably are. That's not going to help you fill the position.

So as the candidate begins to discuss the project (and particularly with me homing in on certain technologies or techniques), I begin to ask more and more specific, and deeper questions. A proficient person will be able to keep up and go along with the conversation. When the candidate can no longer keep up, I've discovered the limit of their knowledge on the given topic. If they can keep up until I no longer have questions, then welcome aboard! I know I can do the job, so if you know as much or more than I do, I can't ask for more. This more "natural" style of communication (as opposed to simple question and answer sessions) also relaxes the candidates a bit more. That helps me filter out any noise caused by nervousness.

Do you have any good interview techniques or pet peeves? Let's hear about it! :-)

Dates, Time Zones, and .NET

What time is it? Simple enough question, but right now, at this exact point in time, there could be several people reading this question in different areas of the world. The answer for each would be different even though it is the exact same temporal moment. Here on planet Earth, the differences all have to do with time zones and we all pretty much know what those are. What's a little more difficult is how that could impact your code.

If someone submits an order and that time is recorded, what time was the order submitted? To answer that, you have to choose a time zone for the display. Times are specified as Zero or Zulu Time (GMT/UTC) + or - some bias according to how far you are from that zone. If you are east of the UTC zone, the bias increases up to the International Date Line. If you are west of the UTC zone, the bias decreases up to the same line. Here in California, we are in the U.S. Pacific zone, which is UTC – 8 hours. So if the order got recorded using UTC time, then we could display that same value here – 8 hours and it would result in the same exact time for the U.S. Pacific zone. A person on the eastern seaboard (let's say N.Y.) would be in UTC – 5, so there the date display would be UTC – 5 hours.

What .NET Does for You

The .NET DateTime type stores not only the date & time, but the bias as well. It is therefore able to translate back and forth from UTC. Since the bias is part of the value, when you serialize and deserialize the value, .NET is also able to calculate the difference in bias between what's in the date/time and what the local time should be, so it sets the date/time value properly for the destination. In other words, let's say a server in N.Y. serializes the date/time value, which is then transmitted to California and deserialized. The value in California would show the same time minus 3 hours. (You can of course, tell serialization to not adjust for time zone bias).

In .NET 2.0 or 3.0, you are also able to easily convert from local time to UTC and vice-versa because .NET knows all about the local bias.

What .NET Doesn’t Do for You

Most of the time zone functionality in the 2.0 and 3.0 BCL somewhat mirrors what's available in the Win32 API. Unfortunately, this translates into – you only get to deal with Local time and UTC. Usually that's not a problem, but if you are dealing with a server-based application that handles data for multiple time zones, you could be a little screwed with 2.0 or 3.0. There's no automatic functionality to translate dates from an arbitrary time zone (that isn’t the local server time zone) to some other arbitrary time zone or UTC. Also, from what I understand, the DateTime type stores the bias, but not the exact time zone (or time zone identifier) which could be important as we'll see in a minute.

How to Get Around

Fortunately, there are a few alternatives. Some 3rd party libraries do expose this functionality based on local or service databases. Alternately, most versions of Windows store alternate time zone information in the registry, allowing you to do all the calculations manually. The information is stored under the HKEY_Local _Machine\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Time Zones key. Each child key represents a different time zone. Most of the values in those time zone keys are self-evident, except the really important one: TZI. TZI is binary data that tells you all about the bias information. You can extract the data by using the BitConverter utility class or unsafe code. The first 4 bytes represent an integer that tells you how many minutes are in the normal bias for the time zone. You might be saying “Minutes? Shouldn't it be hours?” No, there are some time zones that have a 1/2 hour bias, for example. The next value is the Standard Time Bias. This has to do with daylight saving time. When you are NOT in daylight saving time – that is, when you are in standard time – this is an additional bias from the previous normal bias. The value is usually 0 (zero). The next 4 bytes represent an additional bias for daylight saving time (usually 1 hour [60 minutes] if the time zone uses daylight saving time at all). The next bunch of bytes are two SYSTEMTIMEs representing the start date/time of standard time and daylight saving time each year, respectively.

So calculation is pretty straight forward – you can always use the bias and daylight saving time info to convert from a specific time zone to UTC and back, right? Actually, there’s a little more to it than that.

First, there's fallback. This is an edge case that happens on times that fall in the wedges between standard time and daylight saving time during changes. Daylight saving time advances time (usually by an hour) so you end up losing an hour. That day only has 23 hours. On the opposite clock change, you set time back an hour, so you end up with 25 hours in the day, and the same hour could appear twice. If your adjusted time happens to fall into those cracks, you have to take it into account.

Next, you have different daylight saving time periods. There are quite a few time zones and even small regions within a given zone. Some don't use daylight saving time at all. But if they do, daylight saving time only started after a certain year (depending on which zone). If a historical date falls into periods before daylight saving time was established, then it shouldn't be adjusted for DLT. Furthermore, the start and end dates of DLT could change. 2007 in the U.S. is a perfect example. Congress decided to move the change forward a few weeks. So DLT bias calculation prior to October 2007 is different from DLT bias calculation after that date. From a coding perspective, Windows only gives you so much data to work with. You'll notice that some time zone registry keys have a subkey called Dynamic DST. This represents the DLT rule changes, if more than one set of rules apply to that zone, but it is fairly limited in historical context.

Which finally brings us to UTC itself. Coordinated Universal Time (UTC) replaced GMT in 1972. Technically, all date/times prior to 1972 are based on GMT, and all after that period are based on UTC. UTC is based on extremely precise atomic clock adjustments each year, while GMT is not, but for most intents and purposes, the differences are ignored.

Anyway, if you are using .NET 3.5, you are lucky because TimeZoneInfo.FindSystemTimeZoneById gives all this implementation for free (some historical date adjustments notwithstanding). If you're still stuck with 2.0 or 3.0, then you have to take all I covered into account.

 

Type Safety, Variance, and Generics

The goal of a type-safe language is to make sure you don’t shoot yourself in the foot. One of the cornerstones of modeling is the ability to perform substitutions. One form of that is polymorphism – the ability for different objects to accept the same messages. In a type-safe language, the language (compiler) allows you to substitute object types if both object types can accept the same message in some formal way (the important thing being "in some formal way" that the compiler can verify). Many type-safe languages formalize this through defined interfaces. The interface specifies a contract comprising of what members are available, so if two classes implement the same interface, the common interface members are guaranteed to be available on both classes. This principle also applies to a class' "natural" or "default" interface, so that if one class derives from a base class, it is polymorphic with the base class as well, meaning that it can accept all the base class' messages. Let's take a look at a (somewhat) classical example:

C#
public interface IShape
{
    float Height { get; set; }
    float Width {get; set; }
}

public class Rectangle : IShape
{
    // implementation goes here...
}

public class Triangle : IShape
{
    // implementation goes here...
}
VB.NET
Public Interface IShape
    Property Height() As Single
    Property Width() As Single
End Interface

Public Class Rectangle Implements IShape
    ' implementation goes here...
End Class

Public Class Triangle Implements IShape
    ' implementation goes here...
End Class

Because Rectangle and Triangle both implement IShape, they can both guarantee that Height and Width are available. Therefore, it is safe to substitute a reference to IShape with a reference to Rectangle or Triangle, since you can call all of IShape's members from Rectangle or Triangle. For example:

C#
IShape shape = new Rectangle();
shape.Height = 22.5;
shape.Width = 50.2;
or
IShape shape = new Triangle();
shape.Height = 12.8;
shape.Width = 12.8;
VB.NET
Dim shape As IShape = New Rectangle()
shape.Height = 22.5
shape.Width = 50.2
or
Dim shape As IShape = New Triangle()
shape.Height = 12.8
shape.Width = 12.8

And, as I mentioned above, you can do a similar substitution safely with inherited types. For example, in .NET, all classes inherit from Object, so you can substitute a reference to Object with a reference to any class. For example:

C#
object x = new Button();
VB.NET
Dim x As Object = New Button()

We can do this substitution because Button supports all of Object’s members through inheritance. We can call Equals(), ToString(), GetType(), GetHashCode(), etc. from any Button instance.

The reverse, however, isn't true. We can’t safely do the following substitution:

C#
public void DoStuff(object input)
{
    Button btn = input;
    Console.WriteLine(btn.Text);
}
VB
Public Sub DoStuff(input As Object)
    Dim btn As Button = input
    Console.WriteLine(btn.Text)
End Sub

You can't safely substitute Object for Button, because Object doesn't implement all of Button's members. In this case, what happens if we send an integer or a Customer class instance to the method? These types do not implement Text, Height, Width, or any of the other Button-specific members. Therefore, the compiler squawks and stops you from doing the assignment of Object to the Button reference.

For C# and VB.NET (with option strict ON), the rule is that you can safely assign a narrowing type to a more general reference. In other words, if type Y inherits Z (that is, Y : Z), you can safely assign Z = Y. Sometimes, but not always, Z may contain a reference to Y or a Y-derived type (in our example above, that would mean that sometimes someone might actually send a Button through the object input parameter). But it is not safe to assume that, so in order to assign Y = Z you need a specific cast. In other words, Y = (Y)Z, or in VB, you would use a DirectCast or Convert. This is your way of telling the compiler, "yes, I know it’s not safe to do the assignment, but trust me, it will work in this case" (and if it doesn’t, you'll end up with a runtime cast error).

So now with all that in mind, let's talk a little bit about variance and generics. First, the variance part:
Let's say you have types Y and Z, where Y : Z
If the substitution Z = Y is allowed, the types are Covariant.
If the substitution Y = Z is allowed, the types are Contravariant.
If no substitutions are allowed, the types are Invariant.

That brings us to the generics part, which makes it a little more interesting. Currently, all the C# (and VB) generic types are completely invariant, which really sucks. Let's look at a few scenarios.

First, let's use the simple Nullable type as an example.
Nullable defines a HasValue member, and a Value member. The HasValue member is not type-parameter-dependent, but Value is. What that means is that for Nullable<T>, HasValue doesn’t depend on T, but Value does depend on T.

If you have Nullable<int> and Nullable<CustomStruct>, you could call HasValue on both without caring about the specific type parameter. However, you cannot substitute CustomStruct for int (or vice-versa), so you cannot be type-agnostic when calling Value.

You could infer through this little exercise that it would be possible to call all non-type-parameter-dependent members on two generic-derived types (if they came from the same generic template) as if they came from the same "base" class. That is to say, if you have generic template type G<T>, and types Y and Z, and you instantiate G<Y> and G<Z>, both resulting types G<Y> and G<Z> have all the same members from G<T> that are not dependant on Y and Z (the T parameter), and thus you could call all these non-type-parameter-dependent members from both G<Y> and G<Z> in a polymorphic manner.

But you would be wrong. Not because it's impossible or unsafe to do so, but because the CLR/C#/VB guys choose not to include support for doing that. In this case, you have to understand that generic-derived types (G<Y> and G<Z> in this case) are two completely separate classes generated from the same template (and somewhat important to this matter is that they are generated at runtime) – no inheritance or interface mapping is involved. G<Y> and G<Z> are completely invariant. To make that work, you would have to create an IG interface containing all the non-type-parameter-dependent methods, and have G<T> implement that interface. Alternately, you could create a G abstract base class that G<T> inherits. If you implement either of those solutions, you can substitute G<Y> or G<Z> for either IG or G. It's a pain and in some cases (as with ValueTypes) it doesn't work well or at all. There could have been automatic support for this. In fact, I think this would be a very useful addition to generics in .NET.

Secondly, you could infer that for G<Y> and G<Z>, their type-parameter-dependent methods would be covariant if Y : Z. The inference is correct, but again, since the current compilation of generics results in two completely different and independent classes, this won't work either. This is the big pain point most people seem to complain about with regards to generic collections. List<Object> and List<String> are not covariant, so you can't assign a List<String> to List<Object> reference even though Object and String are covariant, especially given the fact that you can put inheritance constraints on type parameters (not that you necessarily need constraints to make this work).

I believe the reason that people intuitively feel they can do this is (and remember that most people use generics for the built-in collections) partly because Array-based types are in fact covariant in .NET. Someone from MS offhandedly told me it was only because they needed to support J#, and Java has covariant arrays. Regardless, the same logic (sadly) doesn't apply to generic collection types or any other generic type for that matter.

So what are some workarounds? I briefly mentioned two above, and I'll expand on it here.

The first is to use make the generic template implement a common interface. For example, if I want to create a generic Field<T> class template, and I want some sort of variance, I need to create something like an IField interface.

C#
public interface interfaceIField : INotifyPropertyChanged, IDataErrorInfo
{
    bool IsChanged { get; }
    string Name { get; }
}

public class Field<T> : IField
{
    public event PropertyChangedEventHandler PropertyChanged;

    private string nameField;
    private bool isChangedField;
    private T valueField;

    public string Name
    {
        get { return nameField; }
    }

    public bool IsChanged
    {
        get { return isChangedValue; }
    }

    public T Value
    {
        get { return valueField; }
        set
        {
            if (Validate(value))
                valueField = value;
            isChangedField = true;
            if (PropertyChanged != null)
                PropertyChanged(new PropertyChangedEventArgs(nameField))
        }
    }

    public abstract bool Validate(T newValue);

    // rest of IDataErrorInfo implementation here
}
VB.NET
Public Interface IField Inherits INotifyPropertyChanged, IDataErrorInfo
    ReadOnly Property IsChanged() As Boolean
    ReadOnly Property Name() As String
End Interface

Public Class Field(of T) Implements IField

    Public Event PropertyChanged As PropertyChangedEventHandler _
        Implements INotifyPropertyChanged.PropertyChanged

    Private nameField As String
    Private isChangedField As Boolean
    Private fieldValueField As String

    Public Property Name() As String
        Get 
            Return nameField
        End Get
    End Property

    Public Property IsChanged() As Boolean
        Get
            Return isChangedValue
        End Get
    End Property

    Public Property FieldValue() As T
        Get
            Return fieldValueField
        End Get
        Set (ByVal value As T)
            If Validate(Value) Then fieldValueField = Value
            isChangedField = True
            RaiseEvent PropertyChanged(New PropertyChangedEventArgs(nameField))
        End Set
    End Property

    Public Mustinherit Function Validate(newValue As T) As Boolean

    ' rest of IDataErrorInfo implementation here
End Class

Now, Field<int>, Field<decimal?>, Field<DateTime>, or Field<anything> have variance through the IField interface, meaning that you can interchangeably call Name, IsChanged, and hook into the events and members of INotifyPropertyChanged and IDataErrorInfo from any instance of any class derived from the Field<T> template through the IField interface. Without IField, you could never do any of that. This would also work if you created a Field abstract base class that Field<T> inherited.

The problem is that this won't work if you create a generic Struct template instead of a generic class template. Structs (or any value type) can't inherit, so you can't use a base class. If you use an interface (like the IField interface above), you run into issues with boxing, which aside from performance, also preclude you from affecting the actual values of the original value type.

This could work just fine without the additional layers of interfaces and inherited base classes if the compilers and the CLR allowed it work – and that would be the only way to make it work with structs. This is somewhat compounded by the fact that both C# and VB.NET force you to create template-derived classes as if you were "inheriting" the template, meaning that you cannot "inherit" any other base classes. However, you don't get any of the polymorphic benefits of inheriting a true base class. So while the CLR guys are somewhat shrugging it off, I think it's a very worthwhile endeavor to add variance to generics.