Hmmm.

Why is SQL called Structured Query Language? You can do so much more with it than simply query. Sorry, just a thought.

CodeProject MVPs

Well, I’ve just been made a CodeProject MVP. This is a great honour for me as I’m among some illustrious company here. There are some truly amazing people in this list, and I guess I’m just there to make the coffee for them.

Let me offer my congratulations to all of the other MVPs and to all of the Microsoft MVPs. You guys and girls really are the best. Thanks also to Chris Maunder and his fantastic team for making CodeProject my second home, it’s really amazing what you’ve accomplished over the last year.

Finally, thanks to everybody who’s posted a question that I could answer or posted a piece of crap that I can pour scorn on. I appreciate it.

Extending Linq.

Sorry for the delay in continuing with the discussions on regular expressions, but I got a little sidetracked with the upgrade to Visual Studio 2008 (.NET 3.5), and all the “goodies” that it brings to you. For the moment I’m going to be putting the discussion on regular expressions to one side. I will get back to them but, for the moment, I want to take you on a journey into .NET 3.5.

Possibly the biggest headline in .NET 3.5 is the introduction of Linq. I have to admit that I was highly skeptical about how useful it will actually be in the “real world”, but having played around with it for a little while I have to admit that I’m actually quite impressed. It’s a well thought out area that actually does do a lot to increase productivity. Now, in this entry I’m not going to delve into the internals of Linq and how to use it. Instead, I’m going to talk about a feature of .NET 3.5 and how it can be used to “extend” the behaviour of Linq. Specifically, we’re going to look at how to use Extension Methods to add the ability to return a DataSet and DataTable from Linq in the same way you’d return a generic list.

The following code shows what you actually need to do to extend the IQueryable object in Linq. For the moment, don’t worry about what IQueryable actually does - once you’ve used Linq for a little while it becomes apparent what this interface is for, and where it fits into the big Linq picture.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.Linq;
using System.Data.Linq.Mapping;
using System.Reflection;
using System.Data.Common;
using System.Data;
using System.Data.SqlClient; namespace LinqTesting
{
 /// <summary>
 /// This class contains miscellaneous extension methods for Linq to Sql.
 /// </summary>
 public static class LinqDataExtensions
 {
  /// <summary>
  /// Add the ability to return a DataTable based on a particular query
  /// to a queryable object.
  /// </summary>
  /// <param name="extenderItem">The IQueryable object to extend.</param>
  /// <param name="query">The query to execute.</param>
  /// <param name="tableName">The name of the datatable to be added.</param>
  /// <returns>The populated DataTable.</returns>
  public static DataTable ToDataTable(this IQueryable extenderItem,
    DbCommand query,
    string tableName)
  {
    if (query == null)
    {       throw new ArgumentNullException("query");
    }
    SqlCommand cmd = (SqlCommand)query;
    SqlDataAdapter adapter = new SqlDataAdapter();
    adapter.SelectCommand = cmd;
    DataTable dt = new DataTable(tableName);
    try
    {
    cmd.Connection.Open();
    adapter.Fill(dt);
    }
    finally
    {
    cmd.Connection.Close();     }
    return dt;
  }
  /// <summary>
  /// Add the ability to return a DataSet based on a particular query
  /// to a queryable object.
  /// </summary>
  /// <param name="extenderItem">The IQueryable object to extend.</param>
  /// <param name="query">The query to execute.</param>
  /// <param name="tableName">The name of the DataTable to be added.</param>
  /// <returns>The populated dataset.</returns>
  public static DataSet ToDataSet(this IQueryable extenderItem,
    DbCommand query,
    string tableName)
  {
    if (query == null)
    {
    throw new ArgumentNullException("query");
    }
    return ToDataSet(extenderItem, query, null, tableName);
  }   /// <summary>
  /// Add the ability to return a dataset based on a particular query
  /// to a queryable object.
  /// </summary>
  /// <param name="extenderItem">The IQueryable object to extend.</param>
  /// <param name="query">A generic dictionary containing the
  /// query to execute along with the name of the table to add it to.</param>
  /// <returns>The populated dataset.</returns>
  public static DataSet ToDataSet(this IQueryable extenderItem,
    Dictionary<string, DbCommand> query)
  {
    if (query == null)
    {
    throw new ArgumentNullException("query");
    }
    if (query.Count == 0)
    {
    throw new ArgumentException("query");
    }
    return ToDataSet(extenderItem, query, null);
  }   /// <summary>
  /// Add the ability to return a dataset based on a particular query   /// to a queryable object.
  /// </summary>
  /// <param name="extenderItem">The IQueryable object to extend.</param>
  /// <param name="query">A generic dictionary containing the
  /// query to execute along with the name of the table to add it to.</param>
  /// <param name="dataSet">An optional DataSet. This allows application
  /// to add multiple tables to the dataset.</param>
  /// <returns>The populated dataset.</returns>
  public static DataSet ToDataSet(this IQueryable extenderItem,
    Dictionary<string,
    DbCommand> query,
    DataSet dataSet)
  {
    if (query == null)
    {
    throw new ArgumentNullException("query");
    }
    if (query.Count == 0)
    {
    throw new ArgumentException("query");
    }
    if (dataSet == null)       dataSet = new DataSet();  

    foreach (KeyValuePair<string, DbCommand> kvp in query)
    {
      dataSet = LinqDataExtensions.ToDataSet(extenderItem,
        kvp.Value,
        dataSet,
        kvp.Key);
    }
    return dataSet;
  }   /// <summary>
  /// Add the ability to return a dataset based on a particular
  /// query to a queryable object.
  /// </summary>
  /// <param name="extenderItem">The IQueryable object to extend.</param>
  /// <param name="query">The query to execute.</param>
  /// <param name="dataSet">An optional DataSet. This allows
  /// application to add multiple tables to the dataset.</param>
  /// <param name="tableName">The name of the DataTable to be added.</param>
  /// <returns>The populated dataset.</returns>
  public static DataSet ToDataSet(this IQueryable extenderItem,
    DbCommand query,
    DataSet dataSet,
    string tableName)
  {
    if (query == null)
    {
    throw new ArgumentNullException("query");
    }
    if (dataSet == null)
      dataSet = new DataSet();
    DataTable tbl = LinqDataExtensions.ToDataTable(extenderItem,
      query,
      tableName);
    if (tbl != null)
    {
      if (dataSet.Tables.Contains(tableName))
        dataSet.Tables.Remove(tableName);
      dataSet.Tables.Add(tbl);
    }
    return dataSet;
  }
 }
}

A couple of points to note about the class. Extension methods have to be in static classes, and they have to be static methods. In order to note what class is being extended, you use the “this” keyword before the first parameter. The code itself is fairly self explanatory and doesn’t do anything clever. The clever bit actually occurs in the Linq side. Here’s an example:

public DataSet GetDataSet()
{
    IQueryable q = GetAllQuery();
    return q.ToDataSet(context.GetCommand(GetAllQuery()), null, "MyTable");
} 

private IQueryable<MyTable> GetAllQuery()
{
    IQueryable<MyTable> q = from p in context.MyTables
                        select p;     return q;
}

The context is the DataContext item that you will create when you import the tables using Visual Studio. And that’s it. Using similar techniques you can extend other parts of Linq as you see fit.

Regular Expressions.

A while ago, I considered writing an article containing some common Regular Expressions. For one reason and another, I never got round to writing this article until somebody recently emailed me to ask if I’d done it or not. Well, feeling guilty, I decided to push ahead with the article. Over the next few days, I’ll be posting about my experiences pulling these regular expressions together and discussing how they hang together.

Anyway, the first regular expression is a simple enough one. It simply grabs all of the tags from a piece of HTML like input:

public static MatchCollection GrabTags(string value)
{
    Regex regex = new Regex(
      @"</?\w+((\s+\w+(\s*=\s*(?:"".*?""|'.*?'|[^'"">\s]+))?)+\s*|\s*)/?>",
          RegexOptions.IgnoreCase
          | RegexOptions.Multiline
          | RegexOptions.IgnorePatternWhitespace
          );
    return regex.Matches(value);
}

Tomorrow, we’ll break this one down into its constituent parts and talk about how it all fits together.

Managed Kernel Transaction Manager - Pt 2.

In my previous posting on using the Kernel Transaction Manager here, I mentioned that I would do a follow up explaining how to ensure that your code behaves on systems that don’t have the KTM present. In other words, how do you ensure that your code runs under pre Vista operating systems? Well, it turns out that this is both very simple and very, very complex at the same time.

Basically, you have the choice of trying to replicate the KTM on older operating systems or you can choose to live with the differences in behaviour between the two areas. If you want the systems to behave exactly the same, then you are going to have to add a lot of code yourself to monitor what’s going on inside the transaction and handle them as appropriate. If you can live with the discrepancy in behaviour, then you can do one of the following two things.

1. You can rely on an exception to tell you that the API method isn’t available in that particular form and fall back to a simpler method in the catch block.
2. You can explicitly test the operating system version to see if it is Vista or later. This is the method that I’m going to take here.

Taking yesterdays example of removing a directory, we can create a method that looks like this:

public static void DeleteDirectory(string path)

{

  if (Environment.OSVersion.Version.Major > 5 && Current.Transaction != null)

  {

    IntPtr txh = null;

    IKernelTx tx = (IKernelTx)TransactionInterop.GetDtcTransaction(Transaction.Current);

    tx.GetHandle(out txh);

    RemoveDirectoryTransacted(path, txh);

  }

  else

  {

    Directory.Delete(path);

  }

}

Note that we don’t bother trying to call the transacted version if the version is less than or equal to 5 and there is no current transaction. There, you’ve now made it so that your code will run on XP. Again, a word of caution. If your logic relies on ACID being enforced in the file system and the KTM isn’t present, you could end up in big trouble indeed, i.e. only rely on ACID if you know that the minimum version that your software runs on will always be Vista.

Managed Kernel Transaction Manager - Pt 1.

Recently I’ve started playing around with Kernel Transaction Manager introduced with Vista. This remarkable (and largely unheralded) piece of technology allows developers to put transaction handling around certain IO based actions such as creating a directory or creating keys in the registry.

The name causes confusion for people because it implies that the transactions can only be used in kernel-mode transactions. This isn’t the case as KTM supports both kernel and user-mode transactions. The name actually means that the transaction engine is built into the Kernel. I suppose they didn’t just call it the Transaction Manager because they have shipped so many different transaction managers over the years that this would just end up confusing people.

First of all the bad news. As KTM exists solely in the kernel, it isn’t available on operating systems prior to Vista. If you use Vista or Server 2008, then you have all you need to start using it.

So, why do I think that transactional file activity is such good news? Well, it allows you to develop systems that follow the ACID principals for areas other than database activity. Suppose that you want to create a directory and write a file into it, but this should only persist if the operation the file depends on completes successfully. The old way of doing this would be to create the directory, write the file and then (if the operation fails), remove the file and directory. That’s a lot of work for you to keep track or, and potentially it’s very error prone. How much better it would be if you could create a transaction around these operations and only commit them if things work as you expect. I suppose an example is in order here.

First of all, you need to “hook” into the API to call the function. Sorry, but there is no managed code equivalent of this in the BCL. You have to wrap the API yourself.

[DllImport("kernel32.dll", SetLastError = true,
  CharSet = CharSet.Auto)]
static extern bool RemoveDirectoryTransacted(
  [MarshalAs(UnmanagedType.LPWStr)]string path,
  IntPtr transaction);

This part of the code is necessary so that you can get access to the kernel transaction. As this is a COM call, you need to import the interface. You can call the interface what you like, the important thing is to use the Guid below, and the method name must be GetHandle.

[ComImport]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
[Guid("79427A2B-F895-40e0-BE79-B57DC82ED231")]
internal interface IKernelTx
{
  void GetHandle([Out] out IntPtr handle);
}

Then, you need to call this:

static void Main()
{
  string path = @"c:\test";
  if (!Directory.Exists(path))
    Directory.CreateDirectory(path);   using (TransactionScope tx = new TransactionScope())
  {
    IntPtr txh = null;
    IKernelTx tx = (IKernelTx)TransactionInterop.
      GetDtcTransaction(Transaction.Current);
    tx.GetHandle(out txh);
    RemoveDirectoryTransacted(test, txh);
  }
}

There are a couple of bits to notice in this sample. First of all, we are using TransactionScope to create the transaction that we are going to work in. Now, you can’t simply pass this into the KTM method. You need to convert it into a pointer that the KTM method can work with (remember that the KTM is unmanaged). Anyway, it’s a simple matter to get the transaction. All you need to do is call the TransactionInterop.GetDtcTransaction for the current transaction. This maps to the COM interface I mentioned above, and you can retrieve the transaction pointer by calling GetHandle. Once you have this pointer, you can pass this into your transacted code.

Now if you run a sample like this, you will notice that the directory is not actually removed. Well, that’s what appears to happen but it’s not quite true. The directory is removed, but the removal is not committed because we haven’t told the transaction to commit. If you don’t commit the transaction then these operations are implicitly rolled back. So, how do you save the changes? Well, simply call Commit() on the transaction, and that will do it. In other words, after the RemoveDirectoryTransacted call, add tx.Commit();.

In a future post, I’ll discuss how you can be a good OS citizen and ensure this code doesn’t fail on older operating systems.

Microsoft Synchronisation Framework

Microsoft is soon to be releasing their Synchronisation Framework. To some people, it’s Microsoft’s answer to Google Gears, but I would have to disagree with them. Simply assessing the two technologies as being like for like misses some fairly fundamental points.

1.  Google Gears is aimed at extending Internet applications onto the desktop. In other words, you can run some fairly sophisticated applications in a browser and have them interact as though they are desktop based. I say appear here because they run in some fairly tight restrictions, such as they are sandboxed. Plus, there is a certain lack of control for the user - where is their data stored, offline, online or some weird combination of the two? Effectively, you can think of Gears as being local storage for web applications.
2. Sync Framework is designed to work the other way round. It’s a desktop based technology so it can run from the desktop up to the server.
3. Sync Framework extends from synchronising items such as folders, emails, databases or pretty much anything else you can think of.

In this example, you can see how easy it is to synchronize file changes.

public void SyncFiles(SyncId sync, SyncId destinationId, string sourceRoot, string destRoot,
  FileSyncScopeFilter filter, FileSyncOptions options
{
  using (FileSyncProvider source = new FileSyncProvider(sync, sourceRoot,
      filter, options))
  {
    using (FileSyncProvider destination = FileSyncProvider(destinationId, destRoot,
       filter, options))
    {
      destination.AppliedChange += new EventHandler<AppliedChangeEventArgs>(OnApplyChange);
      SyncAgent agent = new SyncAgent();
      agent.LocalProvider = source;
      agent.RemoteProvider = destination;
      agent.Direction = SyncDirection.Upload;
      agent.Synchronize();
    }
  }
}  public void OnApplyChange(object sender, AppliedChangeEventArgs args)
{
  if (args.ChangeType == ChangeType.Create)
    Console.WriteLine("The file {0} has been moved to {1}",
      args.OldFilePath, args.NewFilePath);
}

As you can see, synchronising the files is very simple indeed. This level of functionality is available for database content as well. I look forward to its release.

Jools Holland

Well, I went to see Jools Holland in an open air concert at Durham racecourse last night. As usual, he gave an absolutely amazing performance, but it was the single bizarrest experience I’ve had in a long time. For a start, the audience was there with their picnic tables and middle class cars. Bear in mind that Jools Holland played some stuff by the likes of T-Bone Walker which was considered subversive just 30 years ago, it’s now safe (predominately white) middle class music.

Where was the sense of danger? Where was the sense of rebellion?

 Oh - and by the way; if you were in the front row last night. Get a life you complete and utter sad gits - I don’t care if you got there at 4 with your friends Jonquil and Tarquin. You’re sad middle aged people and you’re not bad, mad and dangerous.

Expression Web - Part 1

Well, I’ve just installed Microsoft Expression Web and started to play around with it. I must admit that what I’ve seen so far impresses me. It has a certain reminiscence of Dreamweaver with a hint of CoffeeCub about it. Mind you, the typical Microsoft Gloss is dripping all over it.

Start it up and select New > Web Site then choose a template and you are good to go. The templates are OK, but you are probably going to want to go beyond them quickly. Still, at least they do produce XHTML and CSS code, which is a major improvement on Microsoft’s earlier web efforts FrontPage and Interdev. Now for the good news, there doesn’t seem to be a FrontPage extension anywhere in site.

As I continue to explore the app, I’ll give you an update as to how well it fits into the web development workflow - and whether or not it gets in the way.

Just back from holiday

Well, it’s back to the daily grind after a fantastic holiday in a really charming little holiday cottage in Scotland. The under advertised and under resourced area of Scotland that the cottage was located in, is stunning. While the weather was grotty elsewhere in the UK, the south western corner of Scotland was fantastic.

After spending such a wonderful holiday it’s only fair that I let others know where the cottage is. The cottage (http://www.borrowmoss.com/) is located just outside the book capital of Scotland, Wigtown.