Yield

The yield keyword is one of those things in C# that I thought was very complicated and difficult to understand. I grasped that it was used for creating iterators, but after looking at examples of it's usage I was still confused. Turns out there was something I was missing; when using yield return in a method who's return type is IEnumerable, C# automagically saves the state of the Enumerable, so repeated calls to the method will each return the next value in the Enumerable. With this in mind yield seems pretty straight forward. Reading around there are complexities in it's use, but to be honest I can't be bothered looking into it right now :-) 

I wrote this offensively simple program to prove to myself that I could write some code that used yield:

using System;

using System.Collections.Generic;

namespace Yield

{

    class Program

    {

        static void Main(string[] args)

        {

            foreach (int s in IntSequence())

            {

                Console.WriteLine(s); 

            }

            Console.ReadKey();

        }

        static IEnumerable<int> IntSequence()

        {

            for (int i = 0; i < 10; i++)

                yield return i;

           

            yield break;

        }

    }

}

  

Unsurprisingly it produces the following output:

Book review - Bad Science

To say Bad Science is an interesting book would be an understatement. It's absolutely fascinating. But not only that it's also very enlightening. 

If you have ever read any articles by Ben Goldacre it's quite familiar ground. Homeopathy, the placebo effect, nutritionists, Gillian McKeith, medicine in the media and health scares are a few of the topics covered, in detail. What I really like about the book, which I didn't get from the articles, is the common themes running through the book and commentary on how some of the topics affect our culture and beliefs. Like I said, absolutely fascinating. 

So what about the enlightening stuff I mentioned? Well, it's apparent from reading Bad Science that science is often badly reported by the media. With that being the case, how can find out the truth for yourself? Ben Goldacre helps you out by giving an overview of what to look for in a scientific study; what practices help ensure a fair trial and the kind of tricks people use to give a study the slant they are looking for. He also mentions where you can find publicly available studies. The most useful place I think is the Cochrane Collaboration. This site publishes Cochrane Reviews which are systematic reviews of research. What is a systematic review? From the Cochrane site: "A systematic review is a high-level overview of primary research on a particular research question that tries to identify, select, synthesize and appraise all high quality research evidence relevant to that question in order to answer it". In other words they gather and collate high quality research which allows patterns to be observed that wouldn't be seen in individual small studies. Pretty useful if you ask me.

In summary, if you have any interest in science or have ever read any articles by Ben Goldacre before, I recommend this book. It's a very good read.

Side projects

I've no doubt that many other people have written about this (in fact I sometimes wonder if it's possible to write about a topic that nobody else has already written about) but I think having programming side projects is very important. The chief reason for me is experience. I imagine most programmers are constrained in their day to day jobs. Constrained by frameworks, constrained by time, constrained by practices or maybe constrained by poor code that was created by someone else. There's no such thing as a perfect program so at work there are always areas of the code you think could be improved or should have been done differently. But how do you know that you're right? Simple, try it out for yourself. Structure programs exactly as you want. Try different things. See what works and what doesn't. Having this kind of freedom is an incredibly valuable learning experience. 

There are other reasons for doing side projects as well. I don't know about anyone else but I like trying out new technologies. Working with the same old frameworks and libraries day in day out can become monotonous. Working with new technologies is fun and exciting! Well, most of the time. On the odd occasion you work with something new and it turns out to be a load of rubbish, or not significantly better than something you used previously. Oh well, it's all good experience. 

One piece of advice I would always give to someone who wanted to start a side project is to make something that is going to be used. There's always a danger that any project undertaken in your own time will be left unfinished if you lose interest or motivation. I committed to making the Teesside basketball league website in my spare time and knowing it had to be finished by a certain date helped considerably. Some might argue that it takes the fun out of it, but all I can say is that it helped focus my attention on the features that needed to be implemented rather than the features that I wanted to implement. Without that focus I would still be working on it!

Morning pages

What it's all about

Morning pages is a technique I discovered when reading Pragmatic Thinking and Learning. The idea is to write three pages, by hand, about whatever is in your head first thing in the morning before you do anything else. The idea is to tap into the right side of your brain. What's that? It's the area of your brain that's thought to be responsible for things like intuition, spatial awareness, visual processing (e.g. recognising faces) and non-linear thought. According to the book it operates in an asynchronous manner, you can't control it, and it shuts down when the voice in your head is talking. If you're like me, that's pretty much all the time. The theory is that first thing in the morning you are more in touch with the right side of your brain because you're not fully awake and that little voice hasn't started talking nonsense yet. It could be beneficial to tap into thoughts that come from the right side of your brain as it may provide you with solutions to problems or ideas which have been ticking over unconsciously in your head. In theory anyway. 

What Pragmatic Thinking and Learning says about an asynchronous thought process makes a lot of sense to me. How many times have you thought of an answer to a problem, or remembered something you couldn't recall earlier while you were driving to work or doing the washing up? It happens to me a all the time. For this reason I thought that even if it didn't provide anything useful it would be an interesting exercise, and I'd like to see what comes out of my brain in the morning.

Was it useful?

I have no doubt that it was interesting, but I'm still undecided as to the usefulness of morning pages. One way of measuring the usefulness of an exercise is to compare your expectations with the outcome. What were my expectations? If someone had asked me before I started I would have said "not much", but deep down I think what I really wanted was a great idea or a eureka moment. Well, good news, I've solved the meaning of life, the universe and everything! Not really. My brain isn't special. I would estimate, based on zero measurements, that about seventy percent of what I wrote was nonsense and thirty percent was quite interesting. The interesting stuff did reveal some of the topics I think about, but strangely I didn't write as much about what I would typically think about during the day.

So what did I write about? Reading back through the notes there were a number of main themes. A lot of what I wrote was quite negative. There was a lot of complaining about the fact that nothing was in my head and therefore I had nothing to write about. By the end of two weeks I was getting very annoyed at myself for having an empty morning brain. I also moaned in a very pathetic way about the fact that I was tired, and pondered whether tiredness affected the quality of my notes. To top off my whinging I wondered several times what I would get from doing morning pages.

What about the interesting stuff? On several days the first subject I wrote about was dreams. I don't dream a lot but when I do I tend to remember it for about five minutes after I wake up, so it's no surprise that I wrote about it. Other than that it was very contemplative and very much much about me and my thoughts. One thing is obvious from this exercise; I have a lot of questions in my head, and not very many answers. Questions about my job, questions about the future, questions about life, that kind of thing. 

What kind of things did I write?

Below are a few example of things I wrote. As I said earlier, there was quite a lot of moaning:

"I feel like a broken record"

"I could close my eyes and go back to sleep right now"

"Here we go again. The morning nothing"

"I feel like I've run out of things to write about already" - This was after about four lines on my very first morning

"This feels like a blog. A strange nonsensical blog that nobody would want to read"

"How many times can I write that I'm not thinking about anything?"

"If anyone ever reads this I wonder if they'll think I'm totally mental?"

"It's like my brain doesn't want to think in the morning. My body is awake but my brain stubbornly refuses to get out of bed"

But there was a few interesting sections as well. Having read them back, some of them as quite surprising considering they were written when I was half asleep (which of course is the point):

"Why are some people happier than others? Is it the way they think? Is it the way they approach life? The way they were brought up? A chemical/hormonal imbalance? Can something be done about it?"

"Having some understanding of how other people differ from yourself is very useful. I often wonder what goes through other peoples mind and how they come to their conclusions, especially when they differ radically from my own. But saying that I sometimes wonder how I've drawn my own conclusions in hindsight."

"However much I'm convinced a particular direction is the right one to take, there will always, quite rightly, be arguments and reasons against. The really is no room for dogmatic belief in specific ideas, technologies etc. I think it makes you less open to new ideas and opinions, which isn't good for learning. That's not to say you shouldn't have strong opinions, but you shouldn't force them on other people or be offended if someone thinks you are wrong. Again, see both sides. I'll work on that. It's not easy."

"I wonder how people who are really good at things get that way? Sometimes it seems strangely unattainable. But, does it really matter? Is it important? Who actually cares? Why does it sound like someone is using sandpaper outside? Is everyone aware of how the learn effectively? How do you know which skills are more important? What if you're wrong? How do you overcome cognitive biases? How do you take a more calm, measured approach? How do you if and when you should have kids? What will they look like? How will they behave? How much do people imitate other people? Are people really affected in a noticeable way by when then they were born?"

"How do you make realistic goals? How do you really know what you want? Do goals really help that much? With so much information coming in, how do you filter the useful stuff?"

Book review - Pragmatic Thinking and Learning: Refactor Your Wetware

Conclusion

I know it's a bit strange to start with a conclusion but I just want to come out and say it, Pragmatic Thinking and Learning by Andy Hunt is one of the most interesting and useful books I have ever read. Never before has a book given me so many things to think about, so many new ideas and so many useful tips. 

Let me put that into perspective. There is so much information in this book that was new to me I feel like I've learned a huge amount. Not only that, it also explains some of my own behaviour. It's absolutely fascinating. So much of this book strikes a chord with me I will re-read large amounts of it in the weeks, months and years to come. Saying that, I'm not under any illusion it will be like this for everyone, if you have read similar books, or come across many of the techniques and ideas already then it probably won't be quite as useful. 

What's in it?

Because there is so much content in packed into two hundred and fifty ish pages I'll just include the parts that were most interesting to me. The second chapter talks about the stages between novice and expert referencing the Dreyfus model of skill acquisition. The next few chapters detail how your brain works, including which areas of the brain are responsible for which skills, how to use your brain more effectively by using more of it, and common brain bugs (otherwise known as cognitive biases). Chapter six and seven are all about learning more effectively; setting goals, managing skills, working together, reading techniques, mind maps, dealing with failure and the effects of pressure on learning and thinking. Managing focus is the subject of chapter eight. It covers how to increase focus, managing knowledge, dealing with interruptions, optimising your working environment and staying sharp. Finally, the last chapter has some tips on what to do next. It really does cover a lot of ground.

Each chapter is divided into a number of sections, after each section are a number of "Next Actions". These consist of a number of bullet points which encourage you either think about or act upon the previous section. I find these particularly useful as I can easily fall into the habit of simply reading and not thinking enough. 

Why is it useful?

Let's start with the brain. Pragmatic Thinking and Learning explains how your brain works using various computer metaphors. It describes how your brain is split into the left and right sides, each is responsible for different types of thinking. The left deals with things like logic and language, and the right deals with things like spatial awareness, non-linear thinking, and non-rational thinking. The right side of your brain is asynchronous, you can't control it directly. Have you ever tried to remember the name of a band, song, film, actor, author, book and regardless of have much you try, you simply cannot remember? Then some time in the future, at the least likely moment, in the shower, doing the washing up, driving to work, whatever, it leaps into your mind from seemingly nowhere. I think this must have happened to everybody as some point, but I never knew why. Now I do, that's the right side of your brain. What I find particularly fascinating about this is the possibilities this opens up. Often it's unused and under valved so the techniques suggested by Andy Hunt to get more from the right side of you brain are potentially invaluable. It's certainly something I will try and explore.

The brain learning (as my wife would say) doesn't stop there. Another interesting subject covered is cognitive biases. These are known bugs in how people think. An example of this is the confirmation bias, which is the natural tendency to seek out arguments, facts or opinions that match your own. I think having knowledge of these common faults can make you more aware of your own thoughts, where they have come from and why you are having them, and then question yourself. Is that what I want to think? Is there another side to it? Is there more to it? What is the context? But it's not only the way you think that is important. Often other people's thought processes differ drastically from your own. Sometimes it can be difficult to work with colleagues who are very different from yourself. Chapter five includes an introduction to the Myers Briggs type indicator. This is a model which assigns various 'types' to a person based on their personality. The idea is to gain some insight your own personality type and the personality type of others, which hopefully makes it easier to make to understand and therefore interact with personality types that are different from your own. 

The process of learning is something that can be taken for granted after you leave education. Often not a lot of thought goes into it. I'm certainly guilty of that. It's no good randomly reading a few books or blogs, there is more to consider. What should you learn? What subjects are more important? Why? What kind of learning works for me? How can you make it stick? When should I learn? What do I want to achieve? All of these are valid questions which should be considered so you get the most from whatever you learn. Pragmatic Thinking and Learning discusses all of these questions and offers various techniques and suggestions on how to get the most from whatever you learn. 

Concentration is another issue which I've thought about. Or rather the lack of it. There is a whole chapter dedicated to managing focus. It's pretty important. Everyone has difficulty concentrating now and again so any suggestions on how to increase focus are invaluable. Andy Hunt to the rescue again. He suggests mediation can be very beneficial and suggests various ways to maximise your focus by managing interruptions, distractions and context.

I've only scratched the surface of Pragmatic Thinking and Learning. I'm not convinced I've done a good job of summarising the content of the book, but I hope I've done just enough to pique the interest of anyone who reads this blog. If I have, go and buy it now :-)

Some things I didn't know about Notepad++

Notepad++ has been my text editor of choice for a number of years now. It's got a lot more features now than when I first started using it. I've not really kept track of the additions so I thought it was about time had a good look to see what's available that I've missed. A quick Google search found a lot of useful tips. I stole a few tips from various blogs which I've linked to at the bottom of this post. 

Regular expression search and replace with groups

Okay, this isn't new to me, but it's definitely worth mentioning. Search and replace using regular expressions is extremely useful, even more so when using groups. Different regular expression engines use different syntax for groups, in Notepad++ it's slash and a number.

Here's a stupid example. Say I wanted to add a full stop at the end of each line:

Find what: ^(.*)$

Replace with: \1.

TextFX and Plugins menus

There's all kinds of wonder hidden in these menus. A few of my favourites are:

• Plugins -> Compare (Alt + D) - Compares two files
• TextFX Convert -> HTML Encode
• TextFX Characters -> Proper Case - E.g. "cloud moon goat" becomes "Cloud Moon Goat". Also has upper and lower case options (which are more easily available if you right click in a document)
• TextFX Tools -> Delete line number or first word
• TextFX Settings -> Auto close XML/HTML tags

Custom file extensions

Notepad++ provides highlighting for many different types of file, but misses common file types that I use such as .build and .config. Turns out it's very easy to map custom file extensions. Select Settings -> Style Configurator, then choose the required language and add a space separated list of extensions in the "User ext" box.

Box selection

I was surprised to find out this is exactly the same as Visual Studio. Hold Alt, click and drag to select a column. 

Macros

Macros are something I've never bothered to look into. I assumed they just recorded basic text manipulation (typing, copy, paste, etc.), but turns out they can record more than that. For example you can save search and replace actions which is great for saving any common search and replace operations with complex regular expressions.

Misc

Here's a few other bits and pieces that I thought were worth mentioning:

• Edit -> Clipboard history
• View -> Fold and Unfold all
• Run -> Open containing folder
• Run -> Launch in browser

http://allthingsmarked.com/2009/07/29/associate-file-extensions-with-specific-languages-in-notepad-plus-plus/

http://techbrij.com/518/10-notepad-tips-tricks-fast-development

http://www.technabled.com/2009/05/5-notepad-tips-that-just-work.html

 

Teesside basketball league

What?


The Teesside basketball league (TBL) site stores all the information vital to the smooth running of the Teesside basketball league. It records team information (including match day, tip off time, venue and contact details), league standings, news, fixtures, match results, team statistics, player statistics and generally makes the world a better place.

Why?

I originally created the TBL site in 2009 because I wanted to learn some new skills and apply those skills to a real life project. Primarily I was interested in getting more experience of .NET MVC and unit testing, neither of which I was using at work.

How?

The first version of the site was created in 2009. I was unfortunately forced to use ASP.NET MVC 1, which wasn't the best framework around, but it was considerably better than .NET forms. At some point I stumbled across a something called Sharp Architecture. It is described as a "solid architectural foundation for rapidly building maintainable web applications leveraging the ASP.NET MVC framework with NHibernate". In short it provided me with functionality that, if I had to write myself, would have increased the length of the project by about ten times. Some of the functionality provided included ORM setup, visual studio project templates, easy controller based transaction support and an automatically created in memory database for unit testing queries. In additions it detailed a test driven way of working which was extremely helpful.  

The first version of the site was a lot of hard work, but a very rewarding experience. I got want I wanted, I learned a considerable amount about ASP.NET MVC, SQL Server, jQuery, NUnit and RhinoMocks. Even better I still had a girlfriend (now the wife) at the end of it, which considering the number of evenings I spent with my head buried in my laptop, was a very good thing.

For some reason in 2011 I decided that I was no longer satisfied with MVC 1. I wanted all the new and whizzy features offered by MVC 3. It turned out that Sharp Architecture was a pain in the ass to upgrade to MVC 3 so I'd decided I'd had enough of it. Fortunately I wanted to have a go at implementing some of the features provided by Sharp Architecture so it wasn't a big problem.

The technologies I used in the rewrite included ASP.NET MVC 3, Razor, jQuery, Entity Framework, NUnit, NSubstitute, Ninject, T4 and Chirpy.

In hindsight I would have dropped the Entity Framework and NSubstitute. I now believe the Entity Framework was overkill for this project and something simpler would have sufficed. Unfortunately NSubstitute is missing quite a few features offered by other mocking frameworks so something like Moq or RhinoMocks would have better.

Update - April 2015

Sadly the Teesside basketball league is no more so I've shut down the website. Here are a few pictures of the desktop site, mobile site and app.

 

Saving foreign keys in Entity Framework 4.1

Problem

I've recently experienced a few issues using the Entity Framework. I'm not ashamed to admit I swore in it's general direction on several occasions. This particular problem was updating foreign key fields (called navigation properties in the Entity Framework). I still haven't entirely decided whether Entity Framework is the problem or I'm the problem, but regardless it confused me so I thought I'd write about it.

Example

public ActionResult Edit(Car car, int manufacturerId){     

car.Manufacturer = carService.GetManufacturer(manufacturerId);     

carService.Save(car);

}

Here I'm passing in a complete Car entity except for the foreign key Manufacturer which has to be passed in as a foreign key value and looked up. I assumed that this would work fine, but it doesn't. All the scalar fields are saved, but the navigation property is not. A little research turned up a lot of blogs and stackoverflow questions asking about this problem. It turns out there are two ways of dealing with foreign keys in the Entity Framework. Unfortunately to my mind, they're both rubbish. Firstly there is foreign key association, for which you specify the foreign key column. E.g. ManufacturerId. This means you have to manually load all navigation properties. I don't know about you, but I like less work, not more. The second way of using foreign keys is independent associations. This makes more sense because you reference the foreign key object (e.g. Manufacturer) and these objects can be lazy loaded. Now that's more like it. Unfortunately this seems a little flawed as well. Other people can explain this much better than me, but using an independent association the Car is not aware of any changes to it's navigation properties unless it's attached first. This is why the above example would successfully update all the scalar properties of the Car object but not Manufacturer. 

Less than perfect solution

I found a few solutions to this problem. One was to manually tell the Entity Framework that the navigation property has changed. I really don't like that solution because it seems like unnecessary effort. That, and the fact that I've put hours into an application I've written using generic repositories and it would be a massive pain in the ass to change it all now. Another solution I found was to load the object first, then make the changes to the navigation property. Because the entity is attached, the Entity Framework tracks the changes to the navigation property. There's still a small problem, you have to copy all your changed properties submitted by the user to the object loaded from the database. What?!? I can't be bothered doing that for all my entities! They could have loads of properties. Fortunately I found a few posts mentioning a .NET MVC method called TryUpdateModel. It's pretty poorly documented but what it seems to do is use some kind of magic to copy all matching properties from your view model to the object passed into the method.

Below is my updated example code:

public ActionResult Edit(Car car, int manufacturerId) {     

Car carToSave =  carService.GetCar(car.Id);     

TryUpdateModel(car); // Copy all from car to carToSave          

car.Manufacturer = carService.GetManufacturer(manufacturerId);     

carService.Save(car);

}

As a side note if you view model contains the object you what to save (e.g. MyViewModel.Car) you can pass a "prefix" into the TryUpdateModel method which tells it where to look for matching properties. E.g.

public class MyViewModel {

  public Car Car { get; set; }     

  public int ManfacturerId { get; set; }

}

TryUpdateModel(car, "Car")

Easy form filler for developer testing

For the last few weeks I've been on the look out for some kind of application that will fill out HTML forms. This is purely for testing purposes. There's nothing more annoying that having to repeatedly fill out long forms in order to test some functionality. I wanted something that was no effort at all. E.g. You fill out a form, click save then the next time you come to a page you can somehow load the saved values.

A quick Google search found the form saver plugin for Firefox. Unfortunately it hasn't been updated for ages and doesn't support Firefox 5. Autofill forms was another plugin I came across but it was also no good. It seems to be designed to fill out fairly standard form fields such as addresses, usernames and passwords. It was a pain in the ass to set up for a custom form fields so it didn't appeal to my lazy sensibilities. 

So, out of ideas I sent an email round at work asking if anyone knew of an application that would do the job. A colleague pointed out that Selenium IDE could be the answer. Jackpot! I've used Selenium IDE before, but I've never thought of using it in this way. It's primarily an automated testing tool. The IDE can record a user's actions, store them in a specified language and then reproduce the actions at the click of a button. This makes it an absolutely perfect tool for quick and dirty developer testing. Load the IDE, hit record, fill out the form as desired, save the test case with some kind of meaningful name, then the next time you want to fill out the form, load the test case and hit run. Even better, steps can easily be inserted, edited or removed in the IDE if your form changes, and, the test cases can be saved as files so can easily be shared. Selenium IDE FTW.