Recently I was working with some nullable objects that I needed to add together. This led to some incorrect results when I wrote the code to sum them up due to the use of the null coalescing operator.

The good news is that I wrote the code in a particular way, on purpose, to see what the compiler would do and my intuition proved to be correct. On top of that, I had some unit tests in place, so I was confident that a wrong result would get caught immediately. While I don’t think this is an earth-shattering finding, a quick search shows that the issue has tripped up a few people.

Quick, what’s 2 + 3? Would you expect the answer to be 2? If not, then read on! Here is some code to put this all in perspective.

int? x = 2;
int? y = 3;

int result = x ?? 0 + y ?? 0;
Console.WriteLine(result);  // 2

So why did I end up with a result of two? Clearly 2 + 3 = 5, not 2! The answer is that the precedence rules for the ?? operator played a role in returning the odd result.

After reviewing the C# operators precedence rules, I found that the ?? operator has a very low precedence. The addition operator has higher precedence than it, which essentially causes the entire expression to be evaluated as follows:

// parenthesize based on higher precedence of addition
x ?? (0 + y) ?? 0
x ?? (0 + 3) ?? 0  // y is 3

// the left-hand operand, x, isn't null
// so use its value and ignore the right-hand operand
(x ?? 3) ?? 0

x ?? 0  // once again, ignore the right-hand operand
2       // result is x, which is 2

Bear in mind that these steps may not be exactly how the C# compiler evaluates the expression, but they’re close enough to illustrate how it arrives to the seemingly incorrect result. Eric Lippert’s post titled, “Precedence vs Associativity vs Order,” sheds some light on how the compiler evaluates expressions if you’re looking for more detail.

As you may have guessed, the solution to return the expected result is to group the operator in parentheses so that the intended precedence is maintained. The following code yields the correct result of five:

int? x = 2;
int? y = 3;

int result = (x ?? 0) + (y ?? 0);
Console.WriteLine(result);  // 5, as intended

An alternate approach, applicable with nullable types, is to use the Nullable.GetValueOrDefault method. However, it doesn’t have the brevity of the ?? operator.

int result = x.GetValueOrDefault() + y.GetValueOrDefault();

Most people have an idea of general precedence rules, but as software developers we run into more operators with different levels of precedence. For clarity, and when you’re in doubt, group your expressions in parentheses.

Published by Ahmad Mageed on Software Ninjaneer

Recently a past coworker came across a problem and issued a code challenge to some of her colleagues to help figure it out. This post is my response to that challenge.

Problem and Challenge

In an effort to enhance the readability of a List<T>, especially when nested within another list, the developer decided to implement a new class that inherits from List<T>. This looks like:

public class ProductList : List<Product> { }

She wanted to have an extension method that would operate on the List<T> but return the actual type that inherited from it (ProductList, in this example).

To illustrate, consider an extension method that simply skips the first item in a list and returns the remaining items (exciting, I know). This sample reflects the approach taken:

 
public static class Extensions
{
    public static List<T> SkipFirst<T>(this List<T> input)
    {
        List<T> result = input.Skip(1).ToList();
        return result;
    }
}

The extension method was used in the following manner:

ProductList result = (ProductList)list.SkipFirst();

This caused an InvalidCastException to be thrown:

Unable to cast object of type ‘System.Collections.Generic.List`1[CollectionInheritance.Product]‘ to type ‘CollectionInheritance.ProductList’

At this point she could’ve fallen back on using List<T> but she worked around it using AutoMapper instead. The problem with that approach is that the original intention to provide an elegant solution to be used with other classes following the same inheritance setup was somewhat lost. It now required two steps: (1) calling SkipFirst, and (2) mapping from List<T> to TList.

Dissatisfied, she issued the code challenge seeking an extension method that would work with classes which inherit from List<T> without using AutoMapper.

Understanding why casting fails

Before I get to my solution I wanted to shed some light on why casting doesn’t work. As seen earlier, an explicit cast threw an InvalidCastException. Similarly, usage of the as operator is incorrect and would return a null result.

The reason both approaches fail is because ProductList and List are different types! Yes, they represent the same thing to us on the outside, but they aren’t the same.

Jeffrey Richter touches upon this very topic in his excellent CLR via C# book (p. 287, 3rd ed.):

“… you should never define a new class explicitly for the purpose of making your source code easier to read. The reason is because you lose type identity and equivalence…”

He continues with an example to demonstrate, similar to what follows, which evaluates to false:

bool sameType = typeof(List<Product>) == typeof(ProductList); // false

With that in mind, it’s clear why casting won’t work. Likewise, the as operator returns a null value since, by definition, if the conversion fails it will return null instead of throwing an exception.

Another important point (also raised in the book) is that this approach prevents List<T> from being passed to a method that expects the new class which inherits from List<T> as a parameter. However, going in the opposite direction is allowed. In other words, passing a List<Product> to a method that accepts a ProductList would fail, but passing a ProductList to a method expecting List<Product> is valid since ProductList inherits from List<Product>.

Solution

To meet the challenge I wrote the following extension method:

public static TCollection SkipFirst<T, TCollection>
    (this ICollection<T> input)
        where TCollection : ICollection<T>, new()
{
    var processedItems = input.Skip(1);

    var result = new TCollection();
    foreach (T item in processedItems)
    {
        result.Add(item);
    }

    return result;
}

Usage:

ProductList result = list.SkipFirst<Product, ProductList>();

There you go! Nice and succinct. No casting, and no need for the extra line of code to use AutoMapper. The key to this puzzle was to return TCollection, instead of a List<T>. Furthermore, adding the generic constraints ensures that TCollection implements ICollection<T> and has a public parameterless constructor in order to new up an instance of the class.

Note that List<T> and IList<T> could’ve been used in the extension method.

Implementing custom collections and public APIs

For simplicity’s sake it’s not uncommon to see a subclass of List<T>. Perhaps this is acceptable for internal usage, however it isn’t recommended when exposed in a public API. This issue has been brought up numerous times. If a custom collection needs to have special behavior then it’s possible to change the collection’s implementation if inheriting from classes meant to be extended.

To paraphrase Krzysztof Cwalina, co-author of the .NET Framework Design Guidelines, List<T> wasn’t meant to be extended, unlike Collection<T> which lets you override its members and supports awareness of item modification. Also, List<T> returns a rich set of members that typically aren’t appropriate for all scenarios.

This guideline is also covered by a Code Analysis warning, CA1002: Do not expose generic lists. David Kean, a member of the Code Analysis team, expanded on this warning with supporting code examples in the post titled FAQ: Why does DoNotExposeGenericLists recommend that I expose Collection<T> instead of List<T>?

Published by Ahmad Mageed on Software Ninjaneer

My desktop is hooked up to external speakers, which all audio is emitted from. However, if I want to jump on a Skype call (or similar), all sound is routed to the USB headset as soon as I plug it in. This is great for the duration of the call, but since I have Pandora running almost constantly, or might want to watch (and hear) a video, I don’t want to wear the headset all day. Plus, if I kept it plugged in I might miss all audio cues from Outlook, Skype notifications, and so on. True, most programs have visual cues too, but I might not be looking at that portion of the screen, or at the monitor the application lives on in the case of dual monitors.

I also didn’t want to keep plugging and unplugging the USB, which meant I had to use an accessible USB slot instead of one behind the desktop – let’s face it, who wants to see a distracting bunch of cable near them?

Determined to find a solution to a seemingly simple need, I played around with the sound settings. Originally I thought there might be a way to specify the audio device on a per application basis. Instead, I was happy to find a user-friendly option that’s built right into Windows 7 (maybe Vista too, not sure, but you’re not really using that are you?). Essentially, Windows 7 allows us to specify a default device and a default communications device. Without further ado, here’s how I set it up!

Setting up playback devices

Step 1: Right-click the volume control, and select “Playback devices”

This will bring up the Sound control panel, which will look similar to the following image. On the Playback tab notice that the USB headset is currently set as the default device. Your setup might be different, which is fine since the next couple of steps are the important ones.

Step 2: set your desired speakers as the default device. A green check-mark icon will appear next to it.

Step 3: set the USB headset as the default communication device.

Upon doing so a green phone icon will appear next to it. The final setup should resemble the image below.

That’s it! With the above setup I can now leave my USB headset plugged in, and only use it when speaking to someone on Skype or attending an online presentation. In the meantime all my audio is coming from my external speakers.

Controlling other sounds when placing a call

Windows 7 provides a neat feature that adjusts the sound from other devices when using the communications device. The feature is known as ducking, or stream attenuation. By default, I found that the volume was lowered by 80% when I initiated a Skype call. It’s a reasonable default, but I prefer to completely mute all other sounds. To do so, bring up the Sound control panel again, select the Communications tab, then choose your desired option.

Published by Ahmad Mageed on Software Ninjaneer

During development I’ve had the need to delimit a number of lines with a character. Sometimes these lines are numerous and the task is mind-numbingly tedious if I have to sit there and do it manually.

Imagine you’re working with SQL and have a list of ProductId values on separate lines in your clipboard:

1
12
123
1234

You want to include these values when using the IN keyword:

SELECT * FROM Products
WHERE ProductId IN (...)

The goal is to place commas before the numbers, or after the numbers, as shown below.

-- commas before
SELECT * FROM Products
WHERE ProductId IN (1
,12
,123
,1234)

-- commas after
SELECT * FROM Products
WHERE ProductId IN (1,
12,
123,
1234)

This is the order most people take:

1. Paste numbers between the parentheses
2. Position cursor at the start (or end) of the line with the arrow keys (bad) or Home/End key (better)
3. Insert comma
4. Move to the next line
5. Repeat N – 1 times

There’s got to be a better way to do this right? Right!

The two techniques I’ll cover in this post are:

1. Column mode selection
2. Find/Replace using regular expressions

Column mode selection

The typical selection mode is known as continuous stream selection, which is the default selection via the mouse or is performed by using the Shift + arrow keys. The other mode is column mode, or box mode. It lets you select text in columns with a rectangular portion for vertical editing. Once the columns are selected you have the option of inserting, deleting, copying, or pasting text. It’s also possible to overwrite text (i.e., overstrike) by hitting the Insert key and typing new text.

Some IDEs and text-editors support column mode, such as Visual Studio 2010 and Notepad++. In fact, prior versions of Visual Studio supported it, but they didn’t support text insertion, pasting, or zero-length boxes, which are new features introduced in VS 2010. Thus, it was only good for copying and deleting text. Microsoft SQL Server Management Studio (SSMS, using SQL Server 2008) seems to have the limited pre-VS 2010 support for this, which was the inspiration behind this post’s second technique.

Keyboard approach:

1. Place the cursor at the beginning of the line.
2. Press and hold the Shift + Alt keys, then move the cursor with any arrow key. You will see the cursor flashing on each line you’ve selected.
3. Type the desired character or text to prefix or delimit the lines with. For this example it’s a comma: ,.

Mouse approach:

1. Place the cursor at the beginning of the line.
2. Press and hold the Alt key, then click the left mouse button and move the cursor over the desired text.
3. Modify text as desired.

To clarify, for the purposes of this example the cursor is placed at the beginning of the line. However, column mode can be used anywhere in the text, even in the middle of a word.

With regards to the SQL statement earlier, delimiting at the beginning of the lines is probably the best option when using this feature since the text is lined up properly. The end of your lines usually won’t line up since they might have different lengths. The following screenshot demonstrates selecting the columns and the result after adding commas.

There are a number of places where this technique could be useful. In the earlier screenshot showing the stream and column modes, the Hungarian notation could be eliminated easily. Just select the column of “i” characters and hit Backspace. Voila! Consider another scenario where you want to add or change the access modifier for a bunch of properties on a class, making them all public… column mode to the rescue!

For additional information refer to:

• Notepad++ – Column edit mode
• Visual Studio 2010 – How to: Select and Change Text

Find/Replace using regular expressions

In case your IDE of choice doesn’t support column selection, you can still pull this off if you’re not averse to learning a tiny amount of regex. When I say “tiny” I really mean it. All it takes is one regex metacharacter, and you have two to choose from depending on where you want to place the delimiter.

To solve this task use the IDE’s Find/Replace dialog and select the regex option. The two metacharacters of interest are:

• ^: the caret character matches the position at the beginning of the line
• $: the dollar sign matches the position at the end of the line

These are known as anchors since they match at specific positions.

To place the commas before the numbers, use the caret. Otherwise, use the dollar sign to place them after the numbers. The replacement character will be a comma, and you should select the lines to delimit then restrict the replacement to the active selection.

The following screenshots show the find/replace dialog setup in Microsoft Visual Studio 2010 before the replacement, and the result after the replacement.

Before:

After:

Similarly, it’s possible to place the commas at the end of the lines using the dollar sign metacharacter. The following screenshots demonstrate this approach in Notepad++.

Before:

After:

Beware that while most IDEs and text-editors support regex, some only support a subset of metacharacters or may use different metacharacters completely. The good news is the two metacharacters I’ve introduced are standard amongst most regex flavors, which should make this tip universally applicable. I’ve used this tip in Microsoft Visual Studio, Notepad++, LINQPad, and SSMS.

Note, however, that SSMS exhibits an odd behavior (bug?) when using the caret to delimit at the beginning of the line. It skips the first selected line, resulting in 2 occurrences being replaced instead of the expected 3, per my example. This isn’t such a big deal, since at that point all you need to do is manually do the work on one line. Alternately, to get around this, I include the line before the target line to delimit in my selection. That means including the line that starts with “WHERE” so the replacement skips that line and correctly delimits the remaining lines. That said, SSMS works fine when using the end of line delimiting approach with the dollar sign.

Closing thoughts

Obviously column mode is much simpler than using find/replace and regex. If you have that option, use it. Nonetheless, regex is extremely useful for other scenarios where you want to locate a string or series of characters in a certain pattern, then not only add a prefix or suffix to it, but re-use portions of the original input in the replacement. To do so, knowledge of regex comes in handy and can let you pull off such complex text manipulation tasks. Perhaps that’s a post for another day!

For the purposes of this post, however, I’ve only covered delimiting at the start and end of a string and detailed a comparable one to one solution for the column mode and regex approaches.

Published by Ahmad Mageed on Software Ninjaneer

The advent of C# 3.0 introduced the ability to use implicitly typed local variables via the var keyword. Usage of var has flourished since then, and while some people use it judiciously, others make liberal use of it. Ultimately, most people are bound to run into a scenario where it surprisingly ceases to behave properly. Suddenly using var in a foreach loop causes IntelliSense to stop working properly, and an error is generated upon compiling.

The scenario I’m referring to is enumerating over weakly typed collections. This is common when working with COM, Microsoft Office interop, an ArrayList, and any collection that returns a non-generic IEnumerable (i.e., an IEnumerable<object>).

I’ll use this simple Car class to demonstrate in the upcoming examples:

public class Car
{
    public string Make { get; set; }
    public string Model { get; set; }
    public bool IsHybrid { get; set; }

    public Car(string make, string model, bool isHybrid)
    {
        Make = make;
        Model = model;
        IsHybrid = isHybrid;
    }
}

Now, consider the following example:

var cars = new ArrayList()
{
    new Car("BMW", "335i", false),
    new Car("Infiniti", "G37", false),
    new Car("Toyota", "Prius", true)
};

foreach (var c in cars)
{
    Console.WriteLine(c.Make);
}

As I write the line inside the foreach loop, I immediately notice something fishy. When I type "c." the IntelliSense suggestions pop up, but the properties on the Car class are missing! I expect to see Make, Model, and IsHybrid in the popup, but they are visibly absent. What gives?

Screenshot of IntelliSense and missing properties

Since ArrayList returns an IEnumerable<object> the usage of var results in the variable c being an object type. With this in mind, it makes sense that the only IntelliSense items available are the ones that are defined on all objects.

Ignoring this early warning that something is amiss, attempting to compile the code would generate this error:

‘object’ does not contain a definition for ‘Make’ and no extension method ‘Make’ accepting a first argument of type ‘object’ could be found (are you missing a using directive or an assembly reference?)

Workaround #1: Be explicit

The most straightforward solution is to use an explicitly typed local variable. In this case, we would use Car instead of var. The updated code will make IntelliSense work as expected, and compiles correctly.

foreach (Car c in cars)
{
    Console.WriteLine(c.Make);
}

So what’s the difference and why does this approach work? Allow me to refer to the C# language specification in order to “PhD things up,” as one of my co-workers fondly enjoys accusing me of doing occasionally! Essentially, it works because an implicit cast is occurring within the foreach. Section 8.8.4 of the spec (version 4.0) details how the foreach statement determines the type when var is specified. A bunch of steps take place to determine the type, and they are beyond the scope of this post, however the relevant part of the spec that this falls under occurs when the compiler checks for an enumerable interface. In particular, the spec states:

Otherwise, if there is an implicit conversion from X to the System.Collections.IEnumerable interface, then the collection type is this interface, the enumerator type is the interface System.Collections.IEnumerator, and the element type is object.

The spec also shows the individual components of the foreach statement and its expansion, which clarifies how the casting occurs.

Workaround #2: LINQ to the rescue!

With LINQ we can use the Enumerable.Cast method (or Enumerable.OfType) to cast all the elements of the IEnumerable to a given type. In our case, we will specify a Car type and the result will be a strongly-typed IEnumerable<Car>.

foreach (var c in cars.Cast<Car>())
{
    Console.WriteLine(c.Make);
}

The equivalent query syntax is:

var query = from Car c in cars
            select c;

foreach (var c in query)
{
    Console.WriteLine(c.Make);
}

Notice that the range variable c is explicitly typed in the from clause and omitting the type by writing from c in cars would have generated this compile-time error:

Could not find an implementation of the query pattern for source type ‘System.Collections.ArrayList’. ‘Select’ not found. Consider explicitly specifying the type of the range variable ‘c’.

Normally, when working with data sources which implement IEnumerable<T>, we can omit the type since the compiler infers it.

While the above approaches illustrate the point, the first workaround is the preferred way to deal with this issue unless we plan to do more with the LINQ query. The Enumerable methods are beneficial when constructing a larger LINQ query, allowing us to chain multiple extension methods together. Understanding how extension methods work is vital to appreciate how this all fits together.

In brief, Enumerable.Cast extends the IEnumerable type, which is why it is available for use on the cars ArrayList. The result is a strongly-typed IEnumerable<Car>. Since a large majority of extension methods extend IEnumerable<T>, we can begin to use them after using the Cast or OfType extension methods. Even Enumerable.Select, the most basic of extension methods, isn’t available until one of the aforementioned extension methods is used.

As an example of method chaining, the following query can be used to group hybrid and non-hybrid cars together:

var hybridGroups = cars.Cast<Car>().GroupBy(c => c.IsHybrid);

foreach (var group in hybridGroups)
{
    Console.WriteLine("{0} Cars: {1}",
        group.Key ? "Hybrid" : "Non-Hybrid",
        group.Count());

    foreach (var c in group)
    {
        Console.WriteLine("- {0} {1}", c.Make, c.Model);
    }
}

Published by Ahmad Mageed on Software Ninjaneer

Many new features were listed as part of what’s new in the .NET Framework 4.5 Developer Preview, but the one that caught my eye was this Regex engine update:

Ability to limit how long the regular expression engine will attempt to resolve a regular expression before it times out.

Eager to try this feature out I installed the Visual Studio 11 Developer Preview. For what it’s worth, I initially setup the Windows 8 Developer Preview in VirtualBox and played with Visual Studio 11 there, but that’s not necessary. I later installed the VS 11 Developer Preview on my machine and it behaved nicely as a side by side install with VS 2010.

At the time of this writing this post refers to the .NET Framework 4.5 Developer Preview. The current MSDN documentation about this new feature can be found here. Should the implementation change in the RTM version I will try to update this post accordingly.

Why is this feature helpful?

If you’ve written a few regular expressions the chances are you’ve encountered a pattern that caused the regex engine to take a lengthy amount of time to complete. Some might even claim that the engine hangs and they aren’t going to wait around to find out if a match is finally returned. Despite the excessive processing time you may have wound up with the correct result. However, the time spent may not be acceptable for your application.

Here are a few scenarios where poor regex performance may occur:

• Accepting user supplied input to perform a search on data.
• Accepting user supplied regex patterns to search data.
• Working with large string inputs.
• Using patterns that suffer from excessive backtracking, back-references, and other factors.

The old way of terminating the matching process

Imagine providing a web service that allows people to supply regex patterns to test them against their desired data. How do you prevent someone from maliciously – or accidentally – bringing down your service with a poor pattern? To address this concern developers would need to handle the regex task asynchronously, measure the amount of time elapsed, and abandon the task if it exceeds a reasonable period of time.

Wouldn’t it be great if some timeout mechanism was built-in? Well, now we have just that! This feature addresses Connect feedback from 2007 and it’s great to see Microsoft continue to improve the regex engine.

Show me what’s new already!

Alright! First I’ll cover the new pieces of the Regex class, then I’ll demonstrate with some code.

The .NET 4.5 Regex class has been updated to include:

• New overloaded constructors that accept a matchTimeout parameter of type TimeSpan. This applies to both instances of the Regex class and static usage of the class. If a timeout isn’t specified it will default to the application-wide default timeout value, if set, or the InfiniteMatchTimeout value otherwise.
• Support for an application-wide default timeout value specified by setting the “REGEX_DEFAULT_MATCH_TIMEOUT” property via the AppDomain.SetData method.
• A static InfiniteMatchTimeout field member (TimeSpan) which indicates that a pattern matching operation shouldn’t time out.
• A MatchTimeout property (TimeSpan) representing the time-out interval of the current instance. When using the static methods this property can be accessed by handling the RegexMatchTimeoutException.

Should the matchTimeout be exceeded, or invalid, the following exceptions will be thrown with the given messages:

• RegexMatchTimeoutException: this exception is thrown when a timeout occurs. The message reads, “The RegEx engine has timed out while trying to match a pattern to an input string. This can occur for many reasons, including very large inputs or excessive backtracking caused by nested quantifiers, back-references and other factors.”
• ArgumentOutOfRangeException: according to the class’ metadata this is thrown when “options is not a valid RegexOptions value” or when “matchTimeout is negative or greater than approximately 24 days.” The latter part is what’s new. Although it wasn’t mentioned explicitly, I found that a value of zero is also considered invalid, which should come as no surprise.

Being aware of these two exceptions will be useful when working with the new feature. In addition, handling the TypeInitializationException is useful when using the application-wide timeout default. I’ll demonstrate all of this in the subsequent sections that follow.

Introducing a poor pattern

Consider this pattern: ([a-z ]+)*!

This is a poor pattern with nested quantifiers that will cause excessive backtracking. It specifies a group that should be matched zero or more times (i.e., optional). The group contains a character class to match lowercase alphabets or the space character, one or more times. Finally, an exclamation mark should be matched.

Now, consider this input: “The quick brown fox jumps”

When I tested the pattern with this input it took an average of 11 seconds to process. I tried using the full pangram but it took way too long, so I stopped at the word “jumps” to get an idea of the processing time. As the string grows so will the time needed for the engine to complete it’s pattern matching attempt. This pattern has an exponential complexity of O(2ⁿ).

If you’re interested in the details of what makes this a poor pattern read up on catastrophic backtracking.

The code for the above pattern and input looks like this:

string input = "The quick brown fox jumps";
string pattern = @"([a-z ]+)*!";
bool result = Regex.IsMatch(input, pattern);

Try it out. I’ll wait! Then try it with a longer input and continue reading this post. I suspect it’ll still be running long after you’re done reading.

Limiting processing time with the matchTimeout parameter

Let’s update the code to make it timeout after 4 seconds:

string input = "The quick brown fox jumps over the lazy dog.";
string pattern = @"([a-z ]+)*!";
var matchTimeout = TimeSpan.FromSeconds(4);
try
{
    bool result = Regex.IsMatch(input, pattern,
                                RegexOptions.None,
                                matchTimeout);
}
catch (RegexMatchTimeoutException ex)
{
    // handle exception
    Console.WriteLine("Match timed out!");
    Console.WriteLine("- Timeout interval specified: " + ex.MatchTimeout);
    Console.WriteLine("- Pattern: " + ex.Pattern);
    Console.WriteLine("- Input: " + ex.Input);
}

/* Output:
Match timed out!
- Timeout interval specified: 00:00:04
- Pattern: ([a-z ]+)*!
- Input: The quick brown fox jumps over the lazy dog.
*/

That’s all there is to it! Now the regex engine will give up and throw an exception if 4 seconds elapse. Also notice the helpful properties accessible from the RegexMatchTimeoutException object.

Checking the specified duration with the MatchTimeout property

When using an instance of the Regex class you have access to the MatchTimeout property. In the following example I’ll use a StopWatch and compare the time taken to match a new sentence.

string input = "The English alphabet has 26 letters";
string pattern = @"\d+";
var matchTimeout = TimeSpan.FromMilliseconds(10);
var sw = Stopwatch.StartNew();
try
{
    var re = new Regex(pattern, RegexOptions.None, matchTimeout);
    bool result = re.IsMatch(input);
    sw.Stop();

    Console.WriteLine("Completed match in: " + sw.Elapsed);
    Console.WriteLine("MatchTimeout specified: " + re.MatchTimeout);
    Console.WriteLine("Matched with {0} to spare!",
                         re.MatchTimeout.Subtract(sw.Elapsed));
}
catch (RegexMatchTimeoutException ex)
{
    sw.Stop();
    Console.WriteLine(ex.Message);
}

/* Output:
Completed match in: 00:00:00.0007495
MatchTimeout specified: 00:00:00.0100000
Matched with 00:00:00.0092505 to spare!
*/

Handling invalid matchTimeout durations

What happens when we specify an invalid matchTimeout value? As mentioned earlier, an ArgumentOutOfRangeException will be thrown when matchTimeout is either:

• Negative (see my comments on Regex.InfiniteMatchTimeout next for an exception to the rule)
• Zero
• Greater than approximately 24 days

To see this in action, let’s use the last case. I’m not sure how Microsoft decided on the ~24 day limit, but if you have a regex taking days to process I would question whether you’re using the right tool for the job.

string input = "The quick brown fox jumps over the lazy dog.";
string pattern = @"([a-z ]+)*!";

// invalid timeout of 25 days
var matchTimeout = TimeSpan.FromDays(25);
try
{
    var re = new Regex(pattern, RegexOptions.None, matchTimeout);
    bool result = re.IsMatch(input);
}
catch (ArgumentOutOfRangeException ex)
{
    Console.WriteLine(ex.Message);
}

/* Output:
Specified argument was out of the range of valid values.
Parameter name: matchTimeout
*/

To infinity… and beyond!

Oddly enough the Regex.InfiniteMatchTimeout field has a negative TimeSpan value of -00:00:00.0010000 (-1 ms) that, when passed as a matchTimeout parameter, doesn’t cause the ArgumentOutOfRangeException to be thrown. Talk about being exceptional!

So what exactly is its purpose? As stated earlier it’s used as a default value that indicates a match should not timeout, which is essentially the original pre-.NET 4.5  behavior. It takes effect only when the application-wide default value isn’t set. Thus, it’s quite redundant to pass in as a parameter.

To illustrate, these two regex instances share equivalent timeout values (when no application-wide default value is present):

string pattern = @"\d+";
var re = new Regex(pattern);
var reInfinite = new Regex(pattern, RegexOptions.None, Regex.InfiniteMatchTimeout);
Console.WriteLine(re.MatchTimeout == reInfinite.MatchTimeout);
// Output: True

Application-wide default match timeout

To specify an application-wide default timeout value for all regex operations in an application domain you need to set the “REGEX_DEFAULT_MATCH_TIMEOUT” property. Any regex operation or instance declaration will then use the value assigned to that AppDomain property.

Setting the AppDomain default value

string input = "The quick brown fox jumps over the lazy dog.";
string pattern = @"([a-z ]+)*!";

// AppDomain default set somewhere in your application
var defaultMatchTimeout = TimeSpan.FromSeconds(2);
AppDomain.CurrentDomain.SetData("REGEX_DEFAULT_MATCH_TIMEOUT",
                                defaultMatchTimeout);

// regex use elsewhere...
var sw = Stopwatch.StartNew();
try
{
    bool result = Regex.IsMatch(input, pattern);
    sw.Stop();
}
catch (RegexMatchTimeoutException ex)
{
    sw.Stop();
    Console.WriteLine("Match timed out!");
    Console.WriteLine("Applied Default: " + ex.MatchTimeout);
}
catch (ArgumentOutOfRangeException ex)
{
    sw.Stop();
}
catch (TypeInitializationException ex)
{
    sw.Stop();
    Console.WriteLine("TypeInitializationException: " + ex.Message);
    Console.WriteLine("InnerException: {0} - {1}",
        ex.InnerException.GetType().Name, ex.InnerException.Message);
}

Console.WriteLine("AppDomain Default: {0}",
    AppDomain.CurrentDomain.GetData("REGEX_DEFAULT_MATCH_TIMEOUT"));
Console.WriteLine("Stopwatch: " + sw.Elapsed);

Woah! What’s with all that code?! Most of the code is clear. A default value is set in the AppDomain somewhere in your application. Then a regex operation is used without specifying a matchTimeout value and the RegexMatchTimeoutException will be thrown. When the exception is caught it prints out the applied MatchTimeout, which should be the 2 seconds specified for the AppDomain property. Finally the AppDomain property is displayed, along with the elapsed Stopwatch time. The output should be similar to this:

/* Output:
Match timed out!
Applied Default: 00:00:02
AppDomain Default: 00:00:02
Stopwatch: 00:00:02.0322906
*/

Next, notice the catch block that handles the TypeInitializationException. Re-run the code but this time use a negative (invalid) value for the defaultMatchTimeout variable. When the AppDomain property is invalid the Regex class throws this type of exception and the output of the code above then resembles this:

/* Output:
TypeInitializationException: The type initializer for
'System.Text.RegularExpressions.Regex' threw an exception.
InnerException: ArgumentOutOfRangeException - Specified argument was out of the
range of valid values.
Parameter name: AppDomain data 'REGEX_DEFAULT_MATCH_TIMEOUT' contains an invalid
value or object for specifying a default matching timeout for
System.Text.RegularExpressions.Regex.
AppDomain Default: -00:00:02
Stopwatch: 00:00:00.0550294
*/

Overriding the AppDomain default value

While you may want to use the application-wide default value in most cases, there are times when you want certain regex operations to use different values. Overriding the AppDomain default is as easy as specifying the matchTimeout parameter.

For example:

var defaultMatchTimeout = TimeSpan.FromSeconds(5);
AppDomain.CurrentDomain.SetData("REGEX_DEFAULT_MATCH_TIMEOUT",
                                defaultMatchTimeout);

string input = "The quick brown fox jumps over the lazy dog.";
string pattern = @"([a-z ]+)*!";

var sw = Stopwatch.StartNew();
try
{
    var matchTimeout = TimeSpan.FromSeconds(2);
    bool result = Regex.IsMatch(input, pattern, RegexOptions.None, matchTimeout);
    sw.Stop();
}
catch (RegexMatchTimeoutException ex)
{
    sw.Stop();
    Console.WriteLine("Match timed out!");
    Console.WriteLine("Applied Default: " + ex.MatchTimeout);
}

Console.WriteLine("AppDomain Default: {0}",
    AppDomain.CurrentDomain.GetData("REGEX_DEFAULT_MATCH_TIMEOUT"));
Console.WriteLine("Stopwatch: " + sw.Elapsed);

/* Output:
Match timed out!
Applied Default: 00:00:02
AppDomain Default: 00:00:05
Stopwatch: 00:00:02.0256260
*/

Recognizing the underlying issues

While this new feature is great, it’s important to be cognizant of the underlying issues. Look back at the reasons listed for poor regex performance and ponder over them.

I think this feature is ideal for people who are writing services or features that accept user input. It’s now a breeze to put an end to potentially rampant patterns. A reasonable timeout duration could be set, or it may even be dynamically assigned based on upfront checks of pattern length and input length for flexibility.

That said, if you’re the author of poorly performing patterns then using this as a crutch to fall back on isn’t acceptable. Simply put, take regular expressions seriously and learn to write better patterns by understanding the shortcomings and characteristics of bad patterns.

Conclusion: Default values, existing Regex utilization and guidance

By default the Regex class will not timeout. So what should we do about previous uses of Regex in our projects, if anything? The MSDN documentation states:

We recommend that you set a time-out value in all regular expression pattern-matching operations. For more information, see Best Practices for Regular Expressions in the .NET Framework.

I have some reservations about doing that all the time. I suggest leaving them alone, for the most part. There’s no need to hunt down all your regular expressions just to assign a timeout duration to them. That would entail identifying their current performance in order to avoid setting low timeout durations that would cause them to fail. Instead, focus on the patterns with poor performance, time critical usages, or areas where user input is used. If you’re aware of problem areas, then by all means apply the matchTimeout to them individually after determining reasonable timeout limits.

I would caution against specifying an AppDomain level default timeout and walking away. Again, knowing your patterns and scenarios make a huge difference in what strategy to employ. An AppDomain default is appropriate when you want to set an absolute maximum timeout limit for all your Regex operations to prevent a pattern’s performance from catching you by surprise. With that guard in place you should then identify any patterns you believe need more time and override those values directly. This could also present an opportunity to incorporate unit tests of your patterns to ensure timeouts occur within the expected interval, or that they pass successfully before the limit is reached.

Bear in mind that you’ll want to add exception handling too. Handle the RegexMatchTimeoutException to cover timeouts, and handle the ArgumentOutOfRangeException when dynamically applying a matchTimeout value.

Despite the addition of this feature, most operations execute fairly quickly, usually in the milliseconds to early seconds range. When determining reasonable timeout intervals the MSDN documentation suggests taking the following factors into consideration:

• The length and complexity of the regular expression pattern. Longer and more complex regular expressions require more time than shorter and simpler ones.
• The expected machine load. Processing takes more time on systems with high CPU and memory utilization.

To add to that advice, the length of the input is another factor to consider.

Overall I’m pleased with the addition of this feature and think it provides a simple approach to managing performance concerns without the need of writing ugly workarounds to address application responsiveness.

Published by Ahmad Mageed on Software Ninjaneer