CairoCodeCamp Moments

by KodefuGuru 3. March 2010 22:57

Emad posted a slideshow of photos from CairoCodeCamp. It was one of the most amazing experiences of my life. I not only got to present at an excellent conference, but I was able to visit the pyramids of Giza and the Cairo Museum while I was there! This video will give you a small taste of what I experienced in Egypt.

Tags:

General

Reduce Chain and Extract Projection Refactorings

by KodefuGuru 12. February 2010 17:58

One of the principles I introduce in my new talk is Strive for Functional Cohesion. My original article on the subject focuses on determining where the functionality belongs, then adding it to it’s appropriate place. In that scenario, I had complete access to the code. Sometimes, however, you don’t have access to the code to make the change, or the implementation belongs to an interface. If the functionality is assign appropriately, it makes the code much easier to read.

var lines = File.ReadAllLines("file.txt");
var reversed = lines.Select(s => String.Join(",", s.Split(',').Reverse().ToArray())).ToArray();
File.WriteAllLines("file.txt", reversed);

Although procedural, it is declarative. We’re reading lines, reversing them, then writing them back out. We’ve avoided the morass that is iterative programming here by using LINQ. Despite the benefit from being declarative, it isn’t very readable. In my presentation, I show how by assigning the functionality to the class or interface it belongs makes the code more readable. In this case, an IEnumberable<string> should be able to delimit itself, and it should be able to write itself out.

File.ReadAllLines("file.txt")
    .Select(s => s.Split(',').Reverse().Delimit())
    .Write("file.txt");

This cleanup was accomplished with two, very reusable, extension methods (there’s an overload for Delimit in case it isn’t comma-separated). These extensions are available in Fluent.NET.

public static class EnumerableStringExtensions
{
    private const string Delimiter = ",";

    public static string Delimit(this IEnumerable<string> strings)
    {
        return strings.Delimit(Delimiter);
    }

    public static string Delimit(this IEnumerable<string> strings, string delimiter)
    {
        return string.Join(delimiter, strings.ToArray());
    }

   public static void Write(this IEnumerable<string> strings, string path)
    {
        File.WriteAllLines(path, strings.ToArray());
    }
}

While the application of principles enabled me to write cleaner code, it struck me while I was giving this presentation that I still had to explain what how the reversal occurred. If I have to explain it to an audience, someone is probably having to spend too many brain cycles reading my code to figure out what’s going on. It is easier to figure out that s.Split(‘,’).Reverse().Delimit() reverses a set of delimited strings than String.Join(",", l.Split(',').Reverse().ToArray())).ToArray(), but there must be a way to make the code even easier to read.

Reduce Chain

The first way the statement can be simplified is by using the Reduce Chain refactoring. This takes common links from a method chain and reduces it to one link. In this case, we have 3 methods forming our chain: Split, Reverse, and Delimit. We are operating on the string class, so an extension method will be necessary. Since is a delimited string being reversed, I will call the method DelimitedReverse().

public static class StringExtensions
{
    public static string DelimitedReverse(this string source)
    {
        return source.Split(',').Reverse().Delimit();
    }
}

Any place this common method chain was being called can be reduced to this statement. This adheres to the DRY principle by cleaning up those repeated chains.

File.ReadAllLines("file.txt")
    .Select(s => s.DelimitedReverse())
    .Write("file.txt");

This refactoring is useful if the chain is being called from many different places, but I suspect that I will be making this call only in the context of a LINQ select. There’s another refactoring for that.

Extract Projection

Projection is the term used to describe transforming one sequence of items to another. The LINQ Select statement is the de facto way in C# to perform a projection. In the example, we’re doing a DelimitedReverse() projection on IEnumerable<string>. If we extract the projection and give it a name, the code will be easier to read. If that projection was used in more than one place, then the duplication has been removed as well.

There are two ways to do this. The first way merely passes through to a LINQ Select.

public static IEnumerable<string> DelimitedReverse(this IEnumerable<string> strings)
{
    return strings.Select(s => s.Split(',').Reverse().Delimit());
}

The second way builds an iterator using the yield keyword.

public static IEnumerable<string> DelimitedReverse(this IEnumerable<string> strings)
{
    foreach (var s in strings)
    {
        yield return s.Split(',').Reverse().Delimit();
    }
}

My opinion on which is better? Go with the declarative, pass through version unless the projection is nasty. If your code is easier to read in the iterative version, do it that way.

Here’s how the code now reads.

File.ReadAllLines("file.txt").DelimitedReverse().Write("file.txt");
It’s much better than the original. It’s even easier to read than the version refactored for functional cohesion.

Tags: , ,

Kodefu

Two Microsoft Research Projects to Start the Robot Apocalypse

by KodefuGuru 1. February 2010 19:18

I started writing this article with code in mind. It wasn’t until I put the pieces together that I realized the repercussions. The two projects I write about allow one to specify and create the robots to start the robot apocalypse. Warning: you might need Robotic Studio as well!

Microsoft Research has a tendency to release things that require several cups of coffee to wrap my head around, and the TLA Toolbox is no exception. With it, you write PlusCal algorithms (get it right, it’s an algorithm language, not a programming language) which get’s translated to a TLA+ specification. TLA+ Toolbox can then run a model checker.

To get full usage of the tool, you should read the book, Specifying Systems. It’s available for purchase or free for download. This isn’t your typical evening computer book read though… Specifying Systems is full of mathematical symbols… it reads more like a thesis than a popular computer science book.

HourClockSpec 
Hour Clock Specification formula from Chapter 2 of Specifying Systems

Being the three-letter acronym for three-letter acronyms, I had to sift through a few entries to figure out what TLA actually meant. I eventually found that it stands for temporal logic of actions. It describes behaviors in concurrent systems, which makes sense considering time is an element of concern in concurrent systems.

Mentioning concurrent systems, another project had a release today… SIGMA: Large-Scale and Parallel Machine-Learning Tool Kit. Here’s the summary:

The goal of SIGMA is to provide a group of parallel machine-learning algorithms that can meet the requirements of research work and applications typically with large-scale data or features. The tool kit includes more than 10 algorithms and it makes them run on single multicore machine or on a HPC cluster with hundreds of machines and thousands of CPU cores running.

Parallel machine-learning algorithms? I’ve watched too many science fiction movies to know where this is going.

Installing SIGMA gives you a program called MLUI3 Studio. As promised, there are 10 algorithms including some training data and other inputs for the tests. Some of these algorithms are your typical classification and decision making algorithms. For those of us who fear a robot apocalypse, three stand out.

Neural Networks: with this the possibility exists for all of our machines to combine into one giant processing unit.

Boosting: One processing unit not scary enough? Each individual machine is a poor learning. What if each individual machine on the neural network could provide synergy in boosting other’s learning ability.

Expectation-maximization: The one thing us humans had left going for us was our ability to be random and go with our “gut” instinct.  These instincts are likely the combination of many factors we have trouble computing the correct decision, but our subconscious is lending us a hand (like when you’re in the zone and you do instead of think). With expectation-maximization, we have lost that advantage. The bots can now find the maximum likelihood based on probabilistic models, even when those models depend on unobserved latent variables.

We’re doomed. Cyberdyne is alive… and to think, Microsoft Research has even had a project called Terminator since 2004. How could no one see this coming?

terminatorstudio

Tags: , , , , , , , ,

General

A Tale of Two Principle Violations

by KodefuGuru 24. January 2010 23:57

When I decided to write the With statement in Fluent.NET, I never imagined that I would encounter a common bug and school myself in principles that I happen to present on. But that is what occurred, and I want to share it so others can avoid the same mistakes I made.

With is a Fluent version of Add, only it returns the object or interface it is acting upon rather than void. This allows you to chain methods together in a fluid manner. Instead of calling strings.Add(“Hello”); strings.Add(“World”);, you can call strings = strings.With(“Hello”).With(“World”);. Since you should write tests first, I will post my test that I expected to work.

[TestMethod]
public void With()
{
    var strings = new[] { "This", "is", "a" } .AsEnumerable();
    strings = strings.With("test");

    Assert.IsTrue(strings.Contains("test"));
}

With is cool because it even works with arrays! In fact, the point of it was that it would work with any IEnumerable<T>. Here was my initial attempt at an implementation.

public static IEnumerable<T> With<T>(this IEnumerable<T> source, T item)
{
    var collection = source as ICollection<T> ?? source.ToList();
    
    collection.Add(item);

    return collection;
}

I cast source to an ICollection<T> to get to that more specific interface. If it doesn’t implement ICollection<T> the null coalescing operator is used and source is convert to List<T>. In subsequent calls to With, source would be a List<T> so the casting would work. This violates the Liskov Substitution Principle because I’m operating on an ICollection or converting to one no matter what. I decided that I would knowingly do so because I wanted to extend IEnumerable and optimize for a type that it could convert to itself.

This code is buggy. It compiled, but my unit test failed.

Here’s another example of the same bug.

public static void AddFail(IList<string> strings)
{
    strings.Add("Fail");
}

If you haven’t figured it out, don’t feel bad. 40 people over 2 of my sessions haven’t answered it (out loud anyway) either. I’m certain I’ve seen this failure in many methods over the years as well.

The problem is that arrays implement ICollection<T> and IList<T>. However, arrays are a fixed size. Here’s the error I received when running my test: “System.NotSupportedException: Collection was of a fixed size.”

If you’re performing operations that would modify a collection, you must check the IsReadOnly property of that collection. This fixed the bug and is what I checked into CodePlex… but don’t use it, because it’s still wrong.

public static IEnumerable<T> With<T>(this IEnumerable<T> source, T item)
{
    var collection = source as ICollection<T> ?? source.ToList();

    if (collection.IsReadOnly)
    {
        collection = collection.ToList();
    }
    
    collection.Add(item);

    return collection;
}

It works like a charm, but some things were bugging me about the code. I was talking with some of the guys after the Alabama Code Camp, and I lamented that I was violating the Liskov Substition Principle. We came to the agreement that it’s okay to violate principles if you do so purposefully with full understanding of why you are doing so. But something kept nagging me after I left there and continued the 8 hour drive back home to Columbia. It was more than one principle that I was breaking.

After I spent the night at a motel and was leaving, it suddenly dawned on me that I was truly violating a functional principle from my own C# Ninjitsu talk: Avoid Side Effects. What happens if you already have a list. It’s possible to have unintentional consequences in your code.

[TestMethod]
public void WithShouldNotHaveSideEffects()
{
    var strings = new List<string> { "This", "is", "a" };
    var strings2 = strings.With("test");
    Assert.IsTrue(strings2.Contains("test"));
    Assert.IsFalse(strings.Contains("test"));
}

This test fails because it’s not expected that the original sequence will be modified. It certainly isn’t modified when an array is passed in, why should it work any different with a list?

I knew I was in violation of two different principles: one agile, one functional. The code had to be refactored, so I started thinking about where I went wrong. I decided it wasn’t being enough like LINQ. There is a design guideline, and I’m not sure if it’s written down anywhere: if you return an IEnumerable<T>, Consider Yield. I thought about how I would solve the same problem using yield, and it turn out to be quite simple.

public static IEnumerable<T> With<T>(this IEnumerable<T> source, T item)
{
    foreach (T t in source)
    {
        yield return t;
    }

    yield return item;
}
I’m amazed that I found one common bug and violated two design principles in what started as a three line method. It is rather embarrassing, but I hope by sharing this that you will avoid this bug and apply the Liskov Substitution Principle and Avoid Side Effects in your own code.

Tags: , , , ,

Kodefu

Refactor Switch to Dictionary

by KodefuGuru 20. January 2010 14:48

In my post on Z3, I created an extension method for proving an equation. This extension method used a switch statement to analyze a result and return a string. This is a bad practice: the mapping of one type to another is trapped in code. What if I wanted to move this mapping to a configuration file, a resource file, or a database? Even if I didn’t want to do that, I feel an imperative switch statement isn’t as readable as a declarative statement. Favor Declarative over Imperative. Below is the original code snippet.

public static string Prove(this Context context, Term term)
 {
     context.Push();
     Term not = context.MkNot(term);
     context.AssertCnstr(not);
     switch (context.Check())
     {
         case LBool.False:
             return "valid";
         case LBool.Undef:
             return "unknown";
         case LBool.True:
             return "invalid";
     }
     return null;
 }

When you see case statements simply return or an assign an item, you know you can perform this refactoring. You will create a generic dictionary of both types and use it instead. Here is the refactored example.

public static string Prove(this Context context, Term term)
{
    var checkResults = new Dictionary<LBool, string>
    {
        { LBool.False, "valid" },
        { LBool.Undef, "unknown" },
        { LBool.True, "invalid" }
    };

    context.Push();
    Term not = context.MkNot(term);
    context.AssertCnstr(not);
    
    return checkResults[context.Check()];            
}

The great thing about this is that you have the mappings side by side. Since it is in a dictionary format, it is loadable from another location; the mapping is no longer tied to the code.

Tags: , , , , ,

Kodefu

New Release of Pex and Code Contracts

by KodefuGuru 19. January 2010 16:40

Microsoft Research has released new versions of Pex and Code Contracts. You are required to go to DevLabs to download the commercial evaluation of Pex and Code Contracts. You can retrieve the academic version of Pex and Code Contracts from Microsoft Research.

Pex

Pex v0.21.50115.2 has a few bug fixes in it, but there is a major change to the Stubs framework. It has been renamed to Moles framework. ‘Stubs’ is now named ‘Moles’ and ‘Beaver’ is now named ‘Behaved’. Due to this change, any existing .stubx files will no longer work. Here are the steps to migrate from the previous version of Pex (from the release notes):

  • change the project reference from Microsoft.Stubs.Framework.dll to Microsoft.Moles.Framework.dll
  • rename all .stubx files to .moles, and
    • rename the top <Stubs xml element to <Moles.
    • Change the XSD namespace to http://schemas.microsoft.com/moles/2010/
    • Right click on the .moles file in the Solution Explorer and change the Custom Tool Name to ‘MolesGenerator’.
    • Delete all the nested files under the .moles files
  • Remove references to any compiled .Stubs.dll files in your project
  • In general, remove all .Stubs.dll, .Stubs.xml files from your projects.
  • Rename .Stubs namespace suffixes to .Moles.
  • replace all [HostType(“Pex”)] attribute with [HostType(“Moles”)]
  • in PexAssemblyInfo.cs,
    • rename using Microsoft.Pex.Framework.Stubs to Microsoft.Pex.Framework.Moles
    • rename [assembly: PexChooseAsStubFallbackBehavior] to [assembly: PexChooseAsBehavedCurrentBehavior]
    • rename [assembly: PexChooseAsStubFallbackBehavior] to [assembly: PexChooseAsMoleCurrentBehavior]
  • In general, the ‘Fallback’ prefix has been dropped in the following methods:
    • rename FallbackAsNotImplemented() to BehaveAsNotImplemented()
    • rename class MoleFallbackBehavior to MoleBehaviors
    • rename class StubFallbackBehavior to BehavedBehavors

    Code Contracts

    Code Contracts 1.2.30118.5 is a quick bug fix to address issues with last week’s release. I’m kind of curious about this one fix: “Auto-properties containing && ||, or ? should now be handled properly.” How does an auto-property get one of those symbols… obfuscation?

    Last week’s released dropped support for Silverlight 2 and adds it for Silverlight 4. A new set of reference assemblies have been added to allow contracts on newer APIs. The issues with referencing v3.5 assemblies with contracts from v4 projects are now corrected.

    An interesting development is the ability to use auto properties with contracts. Using invariants, which now must be private, on an automatic property are assigned to pre and post conditions of the compiler generated getter and setter methods (which was the reason for the quick bug fix).

    Tags: ,

    Bleeding Edge

    Team Explorer SP1 Upgrade Fail

    by KodefuGuru 19. January 2010 14:51

    I received an email from a coworker today. She received a new computer and was having trouble viewing work items in Visual Studio. When she tried to open one up, she would receive the following error.

    Could not load type 'Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemTypeDeniedOrNotExistException' from assembly 'Microsoft.TeamFoundation.WorkItemTracking.Client, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

    She had Visual Studio 2008 SP1 installed, and she was obviously connecting to Team Foundation Server. I then decided to look at my version of Team Explorer and compare it to hers. In my installed products (under Visual Studio, Help, About) I had “Microsoft Visual Studio 2008 Team Explorer - ENU Service Pack 1 (KB945140)” and she did not.

    The updates to Team Explorer come from Visual Studio 2008 SP1, so my best guess is that her system had SP1 installed before Team Explorer was installed. If you ever encounter this problem, the fix is to reinstall Visual Studio SP1.

    Tags: , , ,

    General

    Code Camp Raffleboard

    by KodefuGuru 18. January 2010 20:06

    I’m sure one of the most tedious tasks for any conference organizer is to give away hundreds of swag at the end of the conference. By that point, you’re tired and you’re ready to go home. But it’s an essential closer for many people, so we do it any; drawing ticket after blue ticket. There must be another way.

    There is. CodeMash 2.0.1.0 solved this problem with a cool app called raffleboard.

    raffleboard

    Since we’re doing a Code Camp on January 30th, I really wanted this application to help me give away prizes. I checked the web and found that the source code was released. I found it was written in Ruby, which I’ve never actually used before. Luckily, it only took me a few minutes to get everything running: download Ruby, download Sinatra with RubyGems, compile.

    I’m a .NET guy though… I wanted a .NET version of this I could play with. I couldn’t find the license, so I emailed Adam (listed right in the application) and he was cool with me branching it. So now, Code Camp Raffleboard is born. It runs on ASP.NET MVC and uses the Spark View Engine.

    If you’re a Code Camp organizer, or organize other types of conferences, it will make your life easier. Download Code Camp Raffleboard today!

    One warning: neither version runs in Internet Explorer. I will see about fixing this in CCR for a future release. There are problems with the style sheets (too advanced) and jQuery.

    Tags: , ,

    Supplies

    Find and Replace Text with Regex

    by KodefuGuru 12. January 2010 17:18

    A common scenario you may be faced with is displaying text to a user that contains identifying information such as social security numbers. This is a pretty simple task in .NET if you know the correct class and methods to use.

    The first thing you’re going to need is a regular expression pattern. Here’s a simple one for a social security number: \d{3}-\d{2}-\d{4}.

    The next thing you should do is add a reference to System.Text.RegularExpressions. Now, we can get to coding.

    string text = "garbage text 123-12-1234 more garbage";
    string pattern = @"\d{3}-\d{2}-\d{4}";
    text = Regex.Replace(text, pattern, m => 
                        "***-**-" + m.Value.Substring(m.Value.Length - 4, 4));

    Regex.Replace existed before the Func classes, so it requests MatchEvaluator rather than Func<Match, string>. No matter, we can still use a lambda here.

    This was a pretty simple solution to a common problem. Enjoy.

    Tags:

    Kodefu

    Solve SMT Problems with Z3

    by KodefuGuru 11. January 2010 22:06

    Microsoft Research has released Z3 2.4. Z3 is a Satisfiability Modulo Theories problem solver. This is the first time I’ve downloaded Z3 and the first time I’ve researched SMT problems, so I’m unfamiliar with them at this point. Being a little curious as to what implications this could have for the .NET framework, I opened up the dotnet example.

    The it has a simple cs file and a build.cmd, so I built it. When I ran the resulting executable, I’m not sure if I received the expected output because parser_example5 through an exception. The other example wrote formulas to the screen so that was pretty neat. After I opened the cs file, it turns out the example5 was expected to receive an error: “/* the following string has a parsing error: missing parenthesis */.”

    From my brief analysis of this library, it appears that you build an equation with it, then you ask Z3 to prove it. My gripe is the syntax of it. You can build yourself a z3 context easily enough,

    using (Config config = new Config())
    {
        config.SetParamValue("MODEL", "true");
        using (Context context = new Context(config))
        {
    
        }
    }

    but working with it afterwards is a pain to manage. Also, why can’t I take advantage of object initializers to set up the config class? I can understand why you may not want to the context to dispose of the config file in case there are other context’s using the config, but there is some fishy about that the setup.

    Let’s prove something simple. If x==y, then it stands to reason that f(x) == f(y).

    The first thing I’m going to do is write an extension method for the z3 Context class so I can prove a term. I will use the Prove method found in the examples, modified for my usage.

    public static string Prove(this Context context, Term term)
    {
        context.Push();
        Term not = context.MkNot(term);
        context.AssertCnstr(not);
        switch (context.Check())
        {
            case LBool.False:
                return "valid";
            case LBool.Undef:
                return "unknown";
            case LBool.True:
                return "invalid";
        }
        return null;
    }

    Next I’m going to write out the steps necessary to assert that x is equal to y, then ask if fx is equal to fy. This code is contained within the using blocks above.

    FuncDecl f = context.MkFuncDecl("f", context.MkIntSort(), context.MkIntSort());
    Term x = context.MkConst("x", context.MkIntSort());
    Term y = context.MkConst("y", context.MkIntSort());
    Term fx = context.MkApp(f, x);
    Term fy = context.MkApp(f, y);
    context.AssertCnstr(context.MkEq(x, y));
    Term proof = context.MkEq(fx, fy);
    Console.WriteLine("f(x) == f(y): {0}", context.Prove(proof));

    That’s a lot of code, but it is valid! “f(x) == f(y): valid” was returned. But how can we be sure that it won’t always return that? Let’s assert the constraint that x is greater than y.

    context.AssertCnstr(context.MkGt(x, y));

    Now we received the text “f(x) == f(y): invalid.”

    You probably don’t want to write all of that code out. Z3 comes with a few parsers, but I haven’t figured out the syntax for them yet. The methods to call on the context is ParseSmtlibFile, ParseSmtLibString, ParseZ3File, ParseZ3String, ParseSimplifyFile, ParseSimplifyString.

    The code reads to me like C++. Looking at the .NET assembly in reflector, it is just a wrapper over unsafe assemblies. It is a shame, as I would play with it a lot more if it felt like I was using .NET. No worries though: the way things are nowadays it’s a matter of time before I find LinqToZ3 on Codeplex.

    Tags: ,

    Kodefu

    Powered by BlogEngine.NET 1.6.0.0
    Theme by Mads Kristensen

    Whois KodefuGuru

    Chris Eargle

    Chris Eargle
    .NET Community Champion

    LinkedIn Twitter Technorati Facebook

    MVP - Visual C#

     

    INETA Community Champions
    Friend of RedGate
    Telerik .NET Ninja
    Community blogs & blog posts

    I am a #52er


    World Map

    RecentComments

    Comment RSS

    Tag cloud

    Disclaimer

    The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

    © Copyright 2010