Eric Potter

Using C# and the Hungarian Algorithm to Optimize Your Christmas Party Planning

2024-12-21T04:00:00+00:00

This post is part of C# Advent organized by @mgroves.

This is the time of the year when many of us are planning Christmas Parties. There is often a sign-up where attendees can indicate what food they want to bring. These usually ensure people bring the right mix of cookies, pie, and candy canes. But it doesn’t ensure the optimal assignments.

The problem is that these sign-up sheets work on a first-come, first-served basis. For example, maybe Mark was the first one to open the sign-up, and he signed up for pie. When Janet opened the sign-up form, she couldn’t sign up for pie because Mark had already done so. This is a problem because everyone knows that Janet makes the best pie. Seriously, how does she make the pie crust that delicious? Mark makes cookies that are just as good as his pie. So we all want Janet to bring the pie and Mark to bring the cookies.

What we want is a way for everyone to indicate what items they want to bring and assign a score for each item indicating how much they want to bring that item. Then, we can run an algorithm to determine the optimal assignments for who should provide which item.

Assignment algorithms are a surprisingly well-studied topic in computer science. There are algorithms for many different variations for different scenarios. Sometimes, you want to make assignments where both sets have preferences in their pairing. But in our case, only one set has a preference. Janet has a strong preference that she bring the pie. But the pie doesn’t care who bakes it. Although if the pie were sentient, it would absolutely want Janet to bake it. But if the pie were sentient, we’d have moral questions about eating it. Forget I ever mentioned the pie. The pie doesn’t have a preference for the assignment, so we need a one-sided assignment algorithm.

What we want is an implementation of the Hungarian Algorithm. Fortunately, there is a good open source implementation of the Hungarian Algorithm available on Nuget thanks to a user, Vivet.

Modeling the choices

The Hungarian optimizes for an overall “cost.” Each potential pairing is assigned a cost where a lower value means that it is a better choice for the solution than a higher value.

In our example, we want the “cost” of Janet bringing a pie to be low because Janet makes the best pie. As mentioned before, Mark’s cookies and pie are equal, so they get the same “cost.”

Cook	Cookies	PIE	Candy Canes
Mark	3	3	5
Janet	3	1	7
Bob	4	5	5

Using the Hungarian Algorithm with C#

The input is a two-dimensional array that represents the choices. The array doesn’t have any of the label data, just a matrix of the costs. Pass the array to the FindAssignments method.

int[,] costs = {{3, 3, 5}, {3, 1, 7}, {4, 5, 5}};

int[] result = HungarianAlgorithm.FindAssignments(costs);

The result is an array that indicates the optimal pairings. The trickiest part of this whole process is that the data that is returned is a collection of indexes, not the names of the pairs. The input data wasn’t labeled, thus the output values aren’t labeled either. You will have to track them separately.

In our case, the output would be [0,1,2].

Index	Value
0 - Mark	0 - Cookies
1 - Janet	1- Pie
2 - Bob	2 - Candy Canes

Dissecting C# Ranges

2023-12-03T04:00:00+00:00

This post is part of 2023 C# Advent! Be sure to check every day for new posts from the .NET community!

Recently, I was teaching C# to a group of developers. When I got to the topic of ranges, I was surprised at how much nuance there was. In this post, I don’t want to go into all of the features of ranges. That is covered in other places. I want to look at the structure of ranges to see that we can learn about them.

Grab a scalpel, lets slice one open.

Here is an example of a simple range. In case the range is accessing an array of numbers from index 3 to the index 3 from the end.

This works because integer arrays, like many other collections, have a range access property. You can easily add range access to your types by creating a property named ‘this’ that takes a Range as a parameter.

But what exactly is a Range? As you can see from its definition pictured below, it is a struct whose primary pieces of data are a start value and an end value.

But the data type of start and end isn’t int, it is Index. So, what is an index? It is a struct whose primary pieces of data are a value and flag indicating whether or not the value is measured from the end of the collection.

Going back to the original line of code, we can see that the caret is the indicator to the compiler whether or not the fromEnd property on the Index is true.

We can use the Syntax Visualizer in Visual Studio to see how the compiler thinks of Ranges. Range Expressions (1) are wrapped in a BracketedArgumentList. They contain the start expression, the aptly named DotDotToken, and the end expression. In this case, the start expression is a NumericLiteralExpression(2) and the end expression is a IndexExpression(3). As you can see, the IndexExpression contains a CaretToken and a NumericLiteralExpression.

Why doesn’t the start token need to be an IndexExpression? While the compiler can do all kinds of tricks, it helps that the Index type contains an implicit conversion from int to Index.

We have yet to begin to dig into the functionality of Ranges, but I hope this look at their structure gives us a little more understanding of how they work.

Using One Headset with Two Computers

2023-11-27T04:00:00+00:00

Like many of you, I’ve got two computers on my desk when I’m at work. I was getting frustrated that I repeatedly needed to unplug my headset from one computer to hear the audio from the other. Who wants to stop listening to The Mighty Mighty Bosstones on their Mac Mini just so that they can hear part of a tutorial video on their laptop?

I fully realize this is a first-world problem, but I still wanted a better solution. I got a small mixer, hooked it up, and now I can hear the audio from both machines in a single headset. Here’s how I did it.

I bought a Behringer Xenyx 302USB mixer. (disclaimer: I work for Sweetwater, but I don’t benefit from use of these links) It is a relatively inexpensive mixer, and I was able to get one used on Gear Exchange for even less.

The Xenyx has a USB connection and can serve as both a USB input and USB output device. I originally intended to use the USB connection to one computer and send the headphone output from the other computer into the mixer’s second channel. One thing I ran into right away is that USB audio degrades quickly when running through a USB hub. I had to change my setup so the mixer plugs directly into the laptop.

To go from the headphone output into the mixer, I needed an adapter to go from an 3.5mm headphone plug to RCA. The audio seemed better using the RCA connection. So, I got a second breakout cable. I have both computers configured to send the audio output to the headphones, which routes it to the mixer. I use the USB connection, but just to make the mixer a microphone for one of the computers.

I want to point out two caveats. First, the microphone only goes into one of the computers, so I can only really do Zoom calls on one of the machines. This is the behavior I wanted. I just want to make that limitation clear. Second, I already had a headset, the Sennheiser GSP 302, that had the input and output broken out separately. This setup wouldn’t work with a USB headset.

So far, I’m loving this setup. If I need to watch a video on either computer, I can hear it in the same headset. If I want to, I can listen to audio from both computers at once. This could come in handy if I want to add Chumbawumba as the background music to a video tutorial. And I’ve got additional physical volume controls for the headphones and my microphone.

In theory, I could mix audio output from either computer with my mic input and out to a Zoom call, but I don’t see myself doing that. And if I wanted to, I could upgrade my setup to use an XLR mic instead of a headset mic.

It’s been a nice, functional upgrade to my office desk setup. It is a small change that makes my workflow just a little be faster every day.

What I wish I knew when I started using the Microsoft Graph API

2022-12-07T04:00:00+00:00

Note: This post is a part of the 2022 C# Advent Calendar. You can see the rest of the posts here.

The Microsoft Graph API is a powerful way to automate interactions with the Microsoft apps you use every day. For example, you could use it to generate Todo items in your Microsoft TOOD account based on data in an Excel spreadsheet. Or you could respond to an event in your custom application by automatically scheduling an Outlook meeting. You can do all this and more from a single API. Or, if you’re a C# developer, you can do it with a single NuGet package. (packages are also available for other languages)

I like to think of working with the Graph API as being a two step process:

Authenticate
Manipulate

In order to get started working with the Graph API, you have to choose how you are going to authenticate your application and your user. I’m not going to cover authentication in this blog post because it varies so much based on your context and the kind of application you are building. I recommend looking at the official docs to select your authentication provider. More details about how to configure authentication are available in other posts.

Abstract away the authentication details by passing around a GraphServiceClient instance

For the applications I’ve built, I have a class that builds and returns an instance of GraphServiceClient. This means that my authentication provider logic can be encapsulated in that class. The rest of the application interacts with Microsoft Graph via the GraphServiceClient instance.

Explore the Graph API with the Graph Explorer

Microsoft provides a powerful tool called the Graph Explorer to experiment with the available APIs. It can work with dummy data. It can also work with your real data if you authenticate it. The Graph API is expansive. It covers everything from Excel, to Outlook, to OneDrive, and more. Using the Graph Explorer will give you an idea of what is possible.

Checkout the GraphServiceClient.Me Property

If the authentication method you chose authenticates a user and not an application, you can get a lot of valuable information from the Me property of the GraphServiceClient object. For example, you can get the user’s calendar or their ToDo list from properties of this property.

Collections are Paged

Because the lists that are returned could be very long, all collections are paged. If you want to query all of the items in your Outlook calendar, it could be hundreds or thousands of items. When you query the collection, you will only get the first page. You can use the NextPageRequest property of the collection to get the next page.

Summary

This is far from an exhaustive explanation of how to get started with the Graph API. But hopefully, this information will get you started a little faster.

The Graph API enables you to build tools to make your work easier. Have fun exploring all of the possibilities.

Stay Curious.

Quickly Create Test Solutions by Scripting the Dotnet CLI

2022-08-18T04:00:00+00:00

From time to time, I need to create a small C# or F# solution to experiment with a code feature or a library function. Often, what I want to do is create a simple console application and a unit test project that references the console application. This isn’t hard to do in Visual Studio, but it feels like it takes too many steps.

Recently, I read a fantastic new book called Essential F#, by Ian Russel. (You can get it here) In it, he showed that you could quickly create the setup I described above with the dotnet CLI. It was an idea so brilliantly simple that I’m jealous that I didn’t think of it. But I did take it a step further and created a bat file to further automate the process.

The commands in this script are taken directly from the book. The only modification I’ve made is to parameterize the solution name and the project name.

With this script, you could execute something like this command:

CreateFSharpProject SampleSolution SampleProject

This would create a new Solution named SampleSolution. It would contain two projects F#, SampleProject and SampleProjectTests. The test project already has a reference to the primary project and is ready to execute the tests with FsUnit.

I created a similar script file for C# and MSTest.

With these scripts, you can quickly create solutions with the unit test project already configured. That way, you can jump right into the code experiment you want to run.

You can take this concept and modify it to create whatever kind of project and unit test project you want. Hopefully, it is a simple little time saver for you.

Customizing TypeScript String Types with Template Literal Types and Utility Types

2022-08-01T04:00:00+00:00

TypeScript has an interesting feature that lets you define a type for a subset of valid strings. These are called String Literal Types. String Literals are a special kind of Union Type. At first glance, this looks similar to an enumeration. But enumerations use numbers as their underlying storage. String Literals maintains all of the behavior of strings.

If I define a String Literal as I did on line 1 above, I have a type, named Grade, that can only be ‘A’, ‘B’, ‘C’, ‘D’, or ‘F’. So if I try to pass an ‘E’ as an argument to function that takes a Grade as a parameter, I will get a compiler error, as shown on line 8 in the example below.

It is important to remember that these checks are only run at compile time and cannot be used directly for run-time concerns like input validation.

String Literal Types can be very useful when writing DOM manipulation functions. You may have a function that is designed to only work with a subset of tags. You could create a String Literal type for that subset and ensure the function is only called with the appropriate tag names.

Template Literal Types

Template Literal Types, introduced in TypeScript 4.1, take this concept even further. Instead of needing to specify every valid value, they can be generated from String Literals. In the example below, we define a second String Literal called GradeModifiers. Then we use a Template Literal on line 18 to generate every possible combination of Grade and GradeModifier.

As you can see on line 13 in the code example below, we can now enter ‘B+’ as a valid grade, which is great. But we can also pass ‘F-‘, which we don’t want to allow.

As you can see from the Intellisense pictured below, the expandedGrades type that we generated with a Template Literal generate every possible combination, which include ‘F+’ and ‘F-‘. So how do we exclude them?

Utility Types

TypeScript also provides tools to customize type definitions called Utility Types. For our purpose we want to use the Exclude operator, which can remove items from a Union Type. Since String Literals are Union Types, we can use the Exclude operator to exclude ‘F+’ and ‘F-‘ from our list, as pictured below on line 20.

With our function definition updated to use our improved type, we can still pass ‘B+’ as before. But we now get a compiler error if we try to pass ‘F-‘

Thanks to the guy at That Conference who was in my session on Advanced Features of the TypeScript type system that told me about this possible combination. It was a cool moment where I got to learn something in the middle of teaching. Thank you!

A Look Inside the .git Folder

2022-01-30T04:00:00+00:00

Each of the dozens of git repos on your machine contains a .git folder. But you may have never thought about the details of its contents. You know that somehow the folder holds the history of every version of every file ever committed to the repository. You just don’t know how.

The contents are less mysterious than you think. For obvious reasons, git optimizes the contents of the .git folder for size and speed. So you can’t browse into it and see your files. The object files are all named after their guid, and the data is zlib compressed. But the structure and organization is documented and understandable.

I’m not going to go into a full explanation of the files here. Others, like Rob Richardson (blog, twitter) have explained it better than I ever will. It was Rob’s talk at CodeMash that helped me understand how the contents of the .git folder worked. I just created a graphic from the info he shared. Additional details are available at GitReady.com.

I’ll simply summarize by saying that the files can be grouped into five categories:

Objects - (blue)These represent the files and changes. Objects can be further divided into commits, trees, and blobs.
Refs - (red)These are human-readable files that organize the objects
Logs - (green)These are used to quickly generate logs displayed to the user.
Config - (light gray)There are files used to config git’s behavior
Temp - (gray)These are temporary files for information that git needs to hold between command-line actions.

Here is Rob’s definitive guide to what is in the .git folder.

Examining Async Behavior in .NET Notebooks

2021-12-14T04:00:00+00:00

This post was written and published as part of the 2021 C# Advent.

One of my favorite new tools is the .NET Interactive Notebooks plugin for Visual Studio Code. It allows you to intermingle executable code and markdown descriptions of it. So you can explain the code, run the code, and see its results all in one place. This makes the notebooks incredibly useful for teaching and exploring concepts.

One nice feature of Notebooks is that the display function returns an object that can update the displayed value. So unlike console functions that display a list of updated values, Notebooks allow you to update values in place.

I wanted to know what Notebooks could do with asynchronous code. So I created two simple methods that code runs for a long enough period that their asynchronous behavior would be obviously noticeable. Both functions display their current activity and continuously update it in place.

Then I invoke both functions, assign their tasks into variables, and then use Task.AwaitAll to wait until both functions run to completion. When the code is executed, you can see that both functions are executing at the same time. (Yes, I know that they aren’t necessarily running simultaneously at the CPU level.)

Now that we understand the Notebook behavior a little better, we can run some experiments. What happens when we square all the values from 1 to 100 but only cube the values from 1 to 50? We see that the cube function finishes much earlier. What happens when we change the WhenAll function to WhenAny? We see that the squares function stops executing when the cube function completes.

The beautiful part of working with .NET notebooks is that you can experiment with the code right in the notebook. I encourage you to head over to the GitHub repo to download the notebook and try it yourself.

The Checklist - A simple tool to help developers work on complex systems.

2021-11-20T04:00:00+00:00

Earlier this month, a blog post went viral titled “Complexity is killing software developers”. The article is worth reading. It talks about how to control complexity. I agree we should try to reduce complexity. But what do we do when the dev work we do is necessarily complex? In this post, I want to discuss a tool to help developers work more effectively when solving complicated and complex problems.

The Plane Too Complex to Fly In the 1930s, the U.S. Army Air Corps held a competition for manufacturers to design and build the next generation long-range bomber. Boeing’s Model 299 was the fastest, had the longest range, and had the largest payload capacity. But one of its early test flights ended in a tragic crash that killed the pilot and injured the rest of the crew. There were concerns that the new capabilities of the plane made it too complex to fly. But Boeing was able to overcome these challenges, and the plane would go on to have an incredibly successful service. It is now famously known as the B-17 Flying Fortress.

How was Boeing able to deal with the technical complexity of flying such a complicated plane? It wasn’t to reduce the capabilities of the plane or to simplify the cockpit. It was the introduction of a simple checklist. This simple tool reduced the cognitive load on the pilot to the point where it could be operated routinely without any errors.

The Checklist Manifesto In his excellent book, The Checklist Manifesto, Atul Gawande tells the story above and many more like it to demonstrate how checklists can be vital in performing complicated work reliably.

The important thing that he found was that when we are solving complex or complicated problems, we tend to miss simple steps, either because we forget them or we wrongly deem them unimportant. This is not because we are dumb; it is because we are human. Our brains have a limited number of things they can hold in working memory at any given time.

Applications for Developers

How do we apply this information? We need to look for places where we are solving simple problems with a routine set of steps in the midst of a more extensive, more complicated process. It shouldn’t be so common that we can easily have it memorized. It shouldn’t be something that we can automate. (If you can automate it, automate it).

A good example of this would be tasks that only happen periodically on a project. For example, you only periodically need to select cloud-based resources for your production environment. It would be best if you remembered to consider cost, security, performance, and integrations. Each one is important, but you could forget one of them in the excitement of planning new features. Having a checklist would ensure that all of them were at least considered, even if some were easy or not applicable.

Another good example would be onboarding a new developer onto the team. Will you remember to get all of the right tools installed on their laptop? Will you remember to teach them the parts of the domain necessary for the project? Will you introduce them to the appropriate stakeholders?

There are two kinds of checks that are prime candidates for inclusion in a checklist, task checks and communication checks. Task checks are things that must be done. Communication checks are for specific bits of information that need to be communicated to specific people. In your development process, you must correctly identify what needs to be done and who needs to be notified. While developing and deploying an exciting new feature, you might forget that the marketing team needs to be notified when the feature is live on the site. A checklist could prevent this kind of mistake.

Future Tools

One area that I think developers could innovate on the idea of a checklist would be to have tools to generate context-appropriate checklists programmatically. Why have a static checklist when you could generate a checklist specific to your current situation? Maybe your team has a checklist for what you need to do and people you need to notify when you deliver a feature to production. But that list might be different depending on whether or not this feature required database changes. A checklist generator could create a checklist that one had the appropriate checks.

Make a Checklist

Take some time to think about when your team missed a small step in a process. Make a checklist so that it doesn’t happen again. Think about times when your team needs to get a lot of small details right in a short period of time. Make a checklist.

Updating Progress in the Windows Taskbar with C#

2021-06-15T04:00:00+00:00

I’ve been using the Pomodoro Technique to structure my work day for a few years I find it very helpful to plan and focus my work. But it requires a timer. I have used Tomighty and PomoFocus.io, but I wanted something different.

I wanted a way to see the status of the pomodoro at a glance. I decided to try to use the Windows API that allows apps to display their progress in the taskbar to display the progress of a 25 minute pomodoro session.

I wasn’t sure how to update the progress from a Windows Forms app, but then I came across the Microsoft-WindowsAPICodePack Nuget package. It provides two simple methods to manipulate the progress with the taskbar icon.

The first is the SetProgressState, which takes a enum which can be NoProgress, Indeterminate, Normal, Error, or Paused. The normal state puts a green overlay on the icon. I use the Normal state for the first 20 minutes of a pomodro. The Paused state makes the icon yellow, which I use to indicate that the pomodoro has less than 5 minutes remaining. The Error state makes the icon red, which I use to indicate the pomodoro is over.

The second method is SetProgressValue. This takes a current value and a max value and renders the progress bar as an overlay on the icon. I pass the current minute of the pomodoro as the current value and 25 as the max value.

So far, I’ve been pleased with the results. The information is “glancable” without taking up additional real estate on my screen.

If you want to see the code, I’ve posted the repo on GitHub.

What is the opposite of a spell checker?

2021-04-16T04:00:00+00:00

A friend once told me that when I type in front of people, I spell like a drunken toddler on a broken keyboard. For years, I’ve worked to hide my poor spelling ability by removing the errors with a spell check tool.

But then I got a different idea. What if, instead of trying to remove all of my mistakes, I had the software add more. That way, you wouldn’t know which spelling errors were mine and which ones were added by the software.

That is when www.DrunkenToddlerBrokenKeyboard.com was born.

Upgrading Old C# to C# 9: Init Only Setters

2020-12-18T04:00:00+00:00

This is my contribution to the fantastic series of blog posts in this year’s C# Advent Calendar. Please check out the rest of the posts for more great content.

In my past two posts, I’ve looked at upgrading an older C# code base to .Net 5. In this post I’m going to start looking at modernizing the code in that library to use C# 9. Specifically, I’m going to look at the benefits of using Init Only Setters.

In the Biggy code base, there is a class named DbColumnMapping that is a prime candidate for init only setters. The class has properties that hold metadata about columns. Obviously, there is some data, like the table name, that never needs to change. But it is nice to be able to set the value from outside the class.

Here is a place in the class named SqliteDbCore where an object initializer is used to set the TableName property.

You can see how useful it is to be able to set the table name in the object initializer. This could also be done by creating a new constructor for DbColumnMapping, but then the constructor would need to be created and maintained. That wouldn’t be hard, but it would be for work.

In the previous version of the code, the property had a public setter. This functions correctly, but it means that the property can now be legally set at any time, when really the property should be effectively read-only.

This is why C# 9 introduces Init Only Setters. Instead of using the “set” keyword, the property is defined with the “init” keyword.

This allows the property to be set in the object initializer, specifically in this case, in the SqliteDbCore class. But the property cannot be changed after the object creation is done.

This isn’t a feature that will radically change how you write code. It is a feature that will help you clearly express your design decisions to the compiler and to future maintainers of your code.

Upgrading ConfigurationManager for .Net 5

2020-11-24T04:00:00+00:00

In my previous post, I described my efforts to upgrade an older .Net Framework code base to .Net 5. The initial effort was to get some class libraries moved over. Those libraries didn’t have many external dependencies. This post will describe what it took to upgrade other class libraries in the same solution that required dependency changes, many System.Configuration.ConfigurationManager.

While the core Biggy library simply stores objects in JSON files, additional Biggy libraries that store objects as JSON in SQLLite, Postgres, and Azure Blob Storage. There are required Nuget packages for all 3 and luckily for me, all 3 have updated versions that run on .Net 5. So those upgrades were easy.

What was slightly more complex was getting the connection strings for SQLLite and Postgres. The Biggy project started in 2014, and as was considered best practice at the time, the connection strings were stored in app.config files and were accessed with System.Configuration.ConfigurationManager.

With the advent of .Net Core, the best practice is to store settings and connection strings in appsettings.json files. And in a future blog post, I’ll upgrade Biggy to use them. But at this stage in the conversion, I wanted to see if I could continue to use the app.config files.

When I first tried to compile the code, I got a few errors because the compiler couldn’t find the ConfigurationManager type. This because it has been removed from the core set of libraries. While .Net Framework included a plethora of classes in its core set of libraries, .Net Core (and now .Net 5) take a much more modular approach. This makes the core set of libraries much smaller. While the Configuration classes aren’t in the core set of libraries, like many other .Net 5 classes, they are readily available as NuGet packages.

I installed the NuGet package for System.Configuration.ConfiguraitonManager (version 5.0). Right away, the app was able to use the connection strings to connect to the databases successfully.

While I wouldn’t suggest using this NuGet package for new .Net 5 projects, if you upgrade older projects that use app.config or web.config files, this package will get you up and running.

Upgrading a .Net Framework library to .Net 5

2020-10-23T04:00:00+00:00

Next month, .Net 5 will be released. This marks an important evolutionary step for .Net as it switches the primary platform from .Net Framework to .Net Core. Microsoft will upgrade .Net Core 3 and rename it to .Net 5. The version jump signifies that it now supper cedes .Net Framework 4.

It will be important for all .Net Framework projects to move to .Net 5, or start down the slow road to obsolescence. I’ve been a part of multiple professional projects that have upgraded from .Net Framework to .Net Core. In this post (which will be the first of several in this series), I want to take an older open source project and upgrade it to .Net 5, documenting all of the steps along the way.

I am going to upgrade the Biggy library. I choose it because it is roughly 7 years old, making it a more interesting case for upgrading. It is also relatively simple. And I have used it on a side project in the past and I’d honestly like to have to use it on an upcoming side project.

Installing the Tools

I started by installing the preview version of Visual Studio 2019. At the time of this writing, the latest version is 16.8.0 Preview 4. This installed the 5.0.100-rc.2.20479.15 of the .Net SDK. I could have installed the SDK manually and gone through this exercise without Visual Studio. But I also wanted to try out some of the preview features in VS.

Upgrading the Solution and the Project Files

The next step was to upgrade the solution file. In my experience, the easiest way to do this is to copy all of the code to a different folder and create a new solution in the old folder. As long as you use the same file name, your source control history will look like you upgraded the .sln file.

The Biggy solution has multiple projects. I started the conversion by converting the Biggy.Core project, which in turn created the new Biggy solution file.

I made a copy of the Biggy repository. In the primary repository, I deleted all of the files except the README.md file and the .git folder. Then I used Visual Studio to create a new .Net 5 class library. I made sure the solution file was named Biggy.sln and the project file was named Biggy.Core.csproj. I manually edited the csproj file, making the TargetFramework property was set to net5.0.

Then I copied all of the *.cs files from the copy of the project folder to the new project folder.

Upgrading the NuGet packages

The Biggy.Core project had dependencies on 2 NuGet packages, Newtonsoft.JSON and Inflector. I had to upgrade Newtonsoft to a version that supported .Net Core, but that was easy. Inflector doesn’t have a newer version that supports .Net Core. Luckily, the package was only used to pluralize some names. I found a different package named Pluralize.NET.Core that provided the same functionality and it supports .Net Core. Some minor code changes had to be made to switch libraries, but that didn’t take to long.

Once these changes were made, the code compiled and I had a valid .Net 5 assembly.

Upgrading the other projects

After I got Biggy.Core to compile, I went through the same process with Biggy.Data.Json and the Tests project. I did have to make a minor change to the test project because it would check to see if the output directory was either “Debug” or “Release”. By default, .Net Core project put their bin files in directories that end with the platform name. So the debug output was now in “Debug\net5.0” instead of “Debug”.

Results

Overall, moving the first three projects in this solution to .Net Core was pretty easy. If there had been a NuGet package that couldn’t have been easily upgraded or replaced that would have made the process more difficult. In the next post, I’ll try to upgrade Biggy.SqlLite and Biggy.Postges and see if there are larger challenges.

If Unit Tests were Seinfeld Characters

2020-09-27T04:00:00+00:00

As software developers we spend a lot of time writing unit tests. We worry about writing enough tests to give us good code coverage. We obsess over which test framework and test runner to use. We ensure that our tests run quickly. We run our test suite after every change and before each check in.

But do we think about the different types of tests that we need write? Do we ever take the time to think about how our tests complement each other? If our test suites are going to provide maximum value, the tests need to fill different roles and complement each other. We need to be intentional about types of tests we write.

Seinfeld is arguably the best TV show of all time. If you are a fan of the show you love to laugh with Jerry, George, Kramer and Elaine. The writers truly created classic characters. But what took the show to the next level was the brilliance in how the separate characters’ storylines intersected and complemented each other. Let’s look at the role of each Seinfeld character and how we should have unit tests that fulfill similar roles and complement each other in our unit testing just as well as Jerry, George, Kramer and Elaine did on Seinfeld.

Jerry Tests

Jerry was the central character on Seinfeld. Episodes generally revolved around his life. It seemed things always worked out for Jerry. In the episode titled “The Opposite,” Kramer nicknames Jerry ‘Even Steven’ because whenever something went wrong for him, something good happened to balance it out.

Throughout the series Jerry generally enjoyed a successful career and dated an endless stream of beautiful women. Nearly every episode included Jerry obsessing with some obscure fault he discovered in each woman. Even so, Jerry came across as the normal and stable one of the bunch. It is impossible to picture this group of friends without Jerry in it.

As software developers we need to have Jerry tests. We often call these happy path tests. These are the tests that validate that the software behaves as expected when everything goes right. These are the tests that cover the central functionality and the functionality that the software revolves around. We need tests as steady and neurotic as Jerry.

Let’s say that you are writing an application for a client that manufactures widgets. Your application needs to collect data from the machines, aggregate the data about how many widgets are being manufactured per day per machine, and then store the results. Jerry tests would validate that when a normal amount of parts are being manufactured on a normal day, the results are correctly calculated and stored. The Jerry tests are vitally important because they cover the cases the software will encounter the majority of the time. But you do need other kinds of tests.

Every central character with great hair like Jerry’s needs a mess of a sidekick, preferably one without hair. George Castanza’s life and baldness supplied an endless list of problems and failures for Jerry to find to mock.

George Tests

What didn’t go wrong for George? His life was marked by epic failures. George found a way to screw up even the simplest of things.

George specialized in underachieving and working as little as possible. His career included being declared dead by boss George Steinbrenner and posing as a latex salesman for the imaginary company, Vandelay Industries.

George’s love life was a wreck as well. He found a way to implode every good relationship that came his way. The epitome being when his fiancé Susan died from licking poisonous glue on the wedding invitations George picked out because they were cheap. He seemed to fail at every turn.

Our tests suites need to have George tests to reveal failure cases in our software. Some George tests cover little things, like the user entering invalid data. Some George tests cover big things, like failures to find required files. For our software to be robust, it needs to be able to handle errors. If we are going to have confidence in our error handling code it needs to have George tests.

Going back to our example with the manufacturing application, George test would cover the case when one of the machines produces zero parts. Let’s say that you have one class that handles the aggregation and several machine provider classes that handle the communication with the machines. George tests of the aggregation class would cover the cases where the machines providers return null or throw an exception.

George tests will cover common failure modes, but our software also needs to be able to handle the bizarre cases. On Seinfeld the character that encountered and generated the most bizarre situations was Kramer.

Kramer Tests

Kramer was constantly involved in situations that were downright outlandish. How could anyone but Kramer get fired from a job he never had? Or be accused of being a serial killer and not know it? How did he end up driving a ladder truck through New York City?

His friends are constantly bewildered by his bizarre lifestyle. In one episode Kramer found the Merv Griffen set and assembled it in his apartment and lived as if he were on a talk show. In another episode Kramer somehow got mistaken for a pimp while wearing a costume from Joseph And The Amazing Technicolor Dreamcoat. His unexpected antics often screw up his friends’ plans. People never know what to expect with Kramer. He bizarre, unreliable and unpredictable.

When we’re testing our software we are going to need Kramer tests to cover rare cases. These are edge cases. Bizarre cases. The things that one user in a million will try. If our software is going to have a large user base eventually it will encounter the issues that only happen once in a blue moon. Kramer tests give us confidence that blue moons won’t crash our software.

In our example application, Kramer test would test parts per day rates for days that have 23 or 25 hours. (Thanks daylight savings time.) Kramer tests would also cover cases where an abnormally large number of parts are made, possibly enough to overflow an integer.

Not all rare cases are bizarre. Some are just complicated, and our software needs to be able to handle those situations. On Seinfeld, the character who always seemed to end up in overly complicated situations was Elaine.

Elaine Tests

Every aspect of Elaine’s life is unnecessarily complicated. Most of her issues revolve around situations in her job or her love life. Her dilemmas are usually a perfect storm of events caused by her and her friends’ impulsive decisions.

In the episode “Bottle Deposit” Elaine overbids on JFK’s golf clubs for her boss to beat her bra-less rival Sue Ellen Mischke. Then the clubs are stolen when a rogue mechanic takes off with Jerry’s car because Jerry doesn’t want to pay to fix it. Elaine’s only hope then rests on Kramer who is tailing the car through Ohio while driving a mail truck full of cans to recycle in Michigan. Who else, but Elaine could get herself into such a complicated situation?

Elaine tests cover situations were two or three things have to happen together in order for the test case to fail under test. Often, these tests are the ones that require mocks or stubs in order to isolate the code being tested.

In the manufacturing application, Elaine tests would cover situations where multiple instances of the application are running at the same time, potentially generating a race condition. They might also cover cases where the communication with the machines is very slow because of a huge amount of traffic on your network.

Elaine tests can take time to write, especially when you need to generate large datasets or populate complex classes. The result is worth the extra effort though. You can make the work easier by utilizing a mocking framework or implementing the mother or builder pattern to generate your test data. An additional benefit to any of these techniques is that you can reuse these tools in other Elaine tests.

Every Show Needs Extras

These four types of tests are the main types of tests that you will need. In the same way that Seinfeld needed an array of characters to round out each episode, you will need other types of tests. Which types you need will depend on your application.

Do you remember the episode where Japanese tourists misunderstood he conversion from Yen to dollars and ended up sleeping in Kramer’s over-sized dresser? Well, don’t forget you may need tests to cover currency exchange rates.

Who can forget the episode where Elaine’s boyfriend Puddy is an ice hockey psycho fan who wears face paint and scares a poor priest? Remember, you may need tests to cover what data will get visualized.

Puddy and Elaine’s relationship was funny in it’s own right for how many times they broke up and go back together.

do {
	Eliane.Dump(Puddy)
	Elaine.GetBackTogetherWith(Puddy)
} while (season == 9 && (episode == 1 || episode == 2)

Do you need to have a test that will test your software’s behavior if one section of code gets called repeatedly in quick succession?

Another unforgettable was the Soup Nazi. This guy would refuse to serve customers in his restaurant for trivial reasons with his classic catch phrase “No soup for you!” Do you have tests that cover situations where your database connection is denied?

If you only had one kind of test you wouldn’t truly be exercising the system. You need all kinds of tests to give you confidence that the system will work as expected in all kinds of situations. Remember, the fundamental reason we write unit tests is to have confidence in the software. If you don’t have a full cast of tests, you probably have bugs you won’t know about until later.

Serenity now, Insanity later. - Lloyd Braun

Ensemble Cast

A show based on only one of these classic characters from Seinfeld would not be fun to watch for very long. It is in how they related to each other and how their storylines intersected that keeps us watching.

Some of the best moments in the show happened as multiple elements of the characters’ storylines came together.

In “The Marine Biologist” episode, the final scene is amazing because it brings together the story about George pretending to be a Marine biologist to impress his date and Kramer hitting golf balls into the ocean.

“The Apology” episode had a beautiful moment when Kramer’s obsession with staying in the shower as long as possible intersects with Elaine and Jerry’s germaphobe plotline. Happy Ending

Our unit test suites should have this same type of fusion. There should be a set of tests that cover the happy path and a group of tests that cover failure cases. There should also be tests that cover bizarre situations as well as the complicated situations that could occur. In addition, there should be other tests that cover the problem areas specific to your application and your domain. If you have a full cast of tests, hitting ‘Run All’ in your test runner will lead to a happy ending to your development story.

The Importance of Humility in Software Development

2020-08-10T04:00:00+00:00

All too frequently, we hear news of another major software bug. Bugs from big companies make the biggest news. Bugs from little companies seem just as common. As programmers, all these bugs are costing us time and money. We know we need to improve. But we can’t seem to figure out how.

As an industry, we’ve tried solving this problem with increasingly complex processes, increasingly complex tools, and sophisticated techniques. But Edsger Dijkstra proposed a different, simpler solution.

Dijkstra was one of the early pioneers of software development. He made important contributions in compilers, operating systems, and distributed systems. But he didn’t claim that advances in these areas would lead to more reliable software. What he thought would make the most significant difference was humility.

Humility

In his 1972 ACM Turing Award lecture, he described the process he envisioned for consistently delivering high-quality software. He saw the 3 keys to this revolution

The use of abstraction to make programs intellectually manageable
Developing correctness proofs alongside the software
Approaching the task of software development as humble programmers

Why did he think that humility was a critical component? Let’s look at the key passage in the lecture:

Now for the fifth argument. It has to do with the influence of the tool we are trying to use upon our own thinking habits. I observe a cultural tradition, which in all probability has its roots in the Renaissance, to ignore this influence, to regard the human mind as the supreme and autonomous master of its artefacts. But if I start to analyse the thinking habits of myself and of my fellow human beings, I come, whether I like it or not, to a completely different conclusion, viz. that the tools we are trying to use and the language or notation we are using to express or record our thoughts, are the major factors determining what we can think or express at all! The analysis of the influence that programming languages have on the thinking habits of its users, and the recognition that, by now, brainpower is by far our scarcest resource, they together give us a new collection of yardsticks for comparing the relative merits of various programming languages. The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague.

Dijkstra reminds us that our brains are not perfect computing machines. We must not treat them as if they were. We must be open to the possibility at all times that we have made a mistake. Dijkstra is not saying that we should act like we are morons. He’s saying that we need to remember our limitations.

Let me be clear. I’m not trying to reinforce impostor syndrome. I’m not advocating for intellectual gatekeeping. You can be an excellent developer and still need to be mindful of your limitations. In fact, in a true and seemingly paradoxical way, healthy humility can help you go from being a good programmer to a great one.

He says, “brainpower is by far our scarcest resource.” He said that back in 1972. How much more true is that statement today with our modern computers, programming languages, IDEs, and associated tools.

He goes on to explain why our limited brainpower makes well-factored code so critical.

The best way to learn to live with our limitations is to know them. By the time that we are sufficiently modest to try factored solutions only, because the other efforts escape our intellectual grip, we shall do our utmost best to avoid all those interfaces impairing our ability to factor the system in a helpful way.

Application

For developers, we can apply Dijkstra’s advice to be humble in many ways. The most direct application of Dijkstra’s advice is to write code that is as simple as possible. This means that we avoid clever tricks. This means that we use abstraction to simplify code, not make it more complicated. Simplicity improves the maintainability of the code over the life of the application.

Another way that humility benefits developers is that it motivates us to write things down. There is an old Chinese proverb that says, ‘The palest ink is more reliable than the most powerful memory.’ We should not assume that we will remember why we made a decision a certain way months after we made it.

Humility also helps us ask questions when necessary. We need to recognize when we are stuck trying to solve a problem. In those situations, we need to be humble enough to ask for help, putting the need to deliver a good solution above our ego. For experienced developers, this may mean being willing to ask for help from developers with less experience.

Conclusion

In his conclusion, he points out that we need to be mindful of both the complexities of software development and the limitations of our abilities.

We shall do a much better programming job, provided that we approach the task with a full appreciation of its tremendous difficulty, provided that we stick to modest and elegant programming languages, provided that we respect the intrinsic limitations of the human mind and approach the task as Very Humble Programmers.

Examining the fractal nature of coupling and cohesion

2020-07-09T04:00:00+00:00

One of the beautiful things about software development is that you can build any application you want, no matter how large or small, with code. As Fred Brooks so elegantly said “The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination.” We write code to build functions. With more code, we compose the functions into objects or modules, creating a higher level of abstraction. With even more code, we create more levels of abstraction by creating packages or assemblies.

I’ve always been fascinated by the fact that some attributes of the code apply equally at each level of abstraction. In particular, the concepts of coupling and cohesion apply at the function level, the class level, and at the package level.

Coupling is the degree to which a unit of code is dependent on other units of code. Ideally, code has few dependencies. This makes it easier to understand, modify, and fix. In order for the software to do anything meaningful, the separate code units must work together. So our goal is not to eliminate coupling, but to minimize it. Functions can be coupled, modules can be coupled, and packages can be coupled.

If we know that we want to minimize coupling, how do we analyze our code to blocks that might be problematic? We can look at the code itself to find coupling, but it is easier to find it using a visualization tool. Fortunately, C# developers have some good options for tools.

Visual Studio ships with a tool called Code Map that provides a graphical way to look at the dependencies between classes. But the NDepend extension provides an even better way to visualize dependencies with its Dependency Graph tool.

As an example, let’s look at the code for the open-source tool called FancyZones that is a member of the Microsoft PowerToys project. It is a WPF project that enables users to define zones on their screen where they can snap windows.

In this brief video, you can see what the graph looks like as you zoom in and out on the code.

If we look at the graph while zoomed to the level of the namespaces, we can see the coupling between the them.

The arrows represent dependencies. In the example above, you can see that FancyZonesEditor.Converters has a dependency on FancyZonesEditor, which in turn has dependencies on FancyZonesEditor.Utils and FancyZonesEditor.Models. The dependency graph also colors objects based on their relationship to the selected object. In the image above, FancyZonesEditor is red because it is selected. FancyZonesEditor.Converters is green because it is a caller of FancyZonesEditor. FancyZonesEditor.Utils is green because it is a callee. And FancyZonesEditor.Models is pink because has a mutual dependency with FancyZonesEditor.

At this level of magnification we can start to see how the high level structures in this code based are related to one another and how they are coupled. If we zoom in, we can get the same information for the classes.

Now we are looking at all of the classes in the FancyZonesEditor namespace. Notice how the dependency arrows have different widths. NDepend draws the widths proportional to the number of dependent items. You can quickly see that GridEditor has a stronger coupling with GridZone and weak coupling with App. You can use this information to quickly analyze the design of your classes. If there is a dependency that is stronger than it should be, it will be obvious in this view and you can begin to refactor it.

Let’s zoom in further and examine the functions inside of the GridEditor class.

Just like at the other zoom levels we see blocks of code and the dependencies between them. Again we see callers in green and callees in blue.

The thing that is fascinating is how all three images look generally similar. In the same way that the Mandelbrot set or a Jackson Pollock look generally similar at different levels of magnification.

This illustrates the importance of minimizing coupling at all levels. As a developer, you must manage it well in the micro level, the macro level, and all levels in between.

C# Strings with Ranges, and Indexes

2019-12-21T04:00:00+00:00

This blog post is part of Third C# Annual Advent organized by Matt Groves, Developer Advocate Couchbase and Microsoft MVP. Thanks to Matt for giving me an opportunity to participate again this year. You can follow the C# Advent here.

C# 8 introduced the ability to access subsets of collections with range operators. Frequently, ranges are used to access arrays or spans, but they can also be used to access the characters inside a string. This can be especially useful if you know there are fixed length portions of text within the string you are manipulating.

For example, let’s say that you wanted to get the first eight characters of a file name. You could use the Substring method on the string class. Or you could use a range operator to get the first eight characters.

[TestMethod]
public void GetTheFirstEightCharactersOfAString()
{
    string fileName = "myTestFileName.txt";
    string firstEight = fileName[0..8];
    Assert.AreEqual("myTestFi", firstEight);
}

Because the range starts at the beginning of the collection of characters in the string, the first 0 in the range is optional. You could achieve the same this by leaving it out.

[TestMethod]
public void GetTheFirstEightCharactersOfAString_Shorter()
{
    string fileName = "myTestFileName.txt";
    string firstEight = fileName[..8];
    Assert.AreEqual("myTestFi", firstEight);
}

What would be more likely is that you would want to get the file name without the extension. Assuming that you know with certainty that the file extension will always be three characters, you could define a range the starts at the beginning of the string and ends four characters from the end. You do this by using the caret to indicate that an index counts from the back of the collection. So a range of [0..^4] would retrieve all but the last four characters, which in this case would be the file name without “.txt”.

[TestMethod]
public void GetTheFileNameWithoutTheExtension()
{
    string fileName = "myTestFileName.txt";
    string fileNameWithExtension = fileName[0..^4];
    Assert.AreEqual("myTestFileName", fileNameWithExtension);
}

If you wanted to only get the file extension, you could define a range in which both indexes in the range count from the back. A range defined from ^3 to ^0 would get the last three characters in the string. Assuming again that you know that the extension is three characters, this would get the extension.

[TestMethod]
public void GetTheExtension()
{
    string fileName = "myTestFileName.txt";
    string extension = fileName[^3..^0];
    Assert.AreEqual("txt", extension);
}

In the same way that we can omit the number for the beginning of the range if the range starts at the beginning of the collection, we can omit the number at the end of the range if the range ends at the end of the collection. This code would also retrieve the extension of the file, but is slightly shorter.

[TestMethod]
public void GetTheExtension_Shorter()
{
    string fileName = "myTestFileName.txt";
    string extension = fileName[^3..];
    Assert.AreEqual("txt", extension);
}

Lets say that you have a string that is wrapped in curly braces and you only want the text inside the braces. You could easily define a range that leaves off the first and last character of the string.

[TestMethod]
public void GetTextInsideOfBrackets()
{
    string data = "{importantData}";
    string innerData = data[1..^1];
    Assert.AreEqual("importantData", innerData);
}

Using the range operators to access substring of strings doesn’t radically shift the paradigm of your code. But it does make it easier to do some of the operations that we have to write on regular basis. If you have any good examples of how you have used ranges to work with strings, please let me know in the comments below.

It you want to play with the code from this post, you can find it here.

TypeScript Error Handling with Union Types

2019-10-12T04:00:00+00:00

One of the challenges of modern software development is handling errors in elegant ways. Fortunately for TypeScript users, there is a clean way to indicate whether or not a function encountered an error by using Union Types.

Let’s consider an simple example where there is a table in a database that contains information about programmers. We have a function that intends to take the programmer’s name as an argument, lookup the programmer in the database, and return the programmer’s favorite programming language. In an ideal world, the code could look like this. (The actual database lookup has been omitted for brevity.)

import {Programmer} from "./Programmer"

export class ProgrammerRepository{
    public GetByName(name : String) : Programmer{
        // stub for more interesting data retrieval
        return new Programmer("Robert Tables", "TypeScript");
    }
}

// Client Code //

export function GetFavoriteLanguage(name : String) : String{
    var repo = new ProgrammerRepository();
    return repo.GetByName(name).GetFavoriteLanguage();
}

This code is simple and straightforward, but it doesn’t do anything to handle error cases. As any experienced programmer knows, looking up data in a database can fail in a large number of ways.

Throwing Errors

The traditional way to handle failures in JavaScript is to throw Errors. In our simplified example, this would mean that if the repository code couldn’t connect to the database, couldn’t find the record in the database, or some related error, the repository would throw an Error.

import {Programmer} from "./Programmer"

export class ProgrammerRepository{
    public GetByName(name : String) : Programmer{
        // stub for more interesting data retrieval
        throw new Error("data storage error");
    }
}

// Client Code //

export function GetFavoriteLanguage(name : String) : String{
    var repo = new ProgrammerRepository();
    var favLang : String;

    try{
        favLang = repo.GetByName(name).GetFavoriteLanguage();
    } catch {
        favLang = "Could not be found"
    }
    return favLang;
}

This implementation of GetByName will throw an Error if the lookup fails for any reason. This technique functions properly and many developers are used to this style of error handling.

One potential problem with this style of error handling is that the client code is not forced to handle the error. The client code could ignore the potential exception, which could lead to unhandled exceptions in production.

A second potential problem is that the signature of the GetByName function does not indicate what kinds of errors might be thrown.

Returning a Union Type

The TypeScript type system supports a mechanism called Union Types. Union Types specify that an object will be instance of one of the types that are joined together in the union.

On line 4 of the following example, a union type named ProgrammerLookupResult is defined as the union of Programmer and LookupFailed. All of the instances of ProgrammerLookupResult will either be an instance of a Programmer or an instance of the LookupFailed class.

import {Programmer} from "./Programmer"
import {LookupFailed} from "./LookupFailed"

export type ProgrammerLookupResult = Programmer | LookupFailed;

export class ProgrammerRepository{
    public GetByName(name : String) : ProgrammerLookupResult {
        // stub for more interesting data retrieval
        return new LookupFailed("Entity not found");
    }
}

// Client Code //

export function GetFavoriteLanguage(name : String) : String{
    var repo = new ProgrammerRepository();
    var favLang : String = "";

    let result = repo.GetByName(name);
    
    if(result instanceof Programmer){
        favLang = result.GetFavoriteLanguage();
    } else if (result instanceof LookupFailed) {
        favLang = "Could not be found";
    }
    return favLang;
}

This implementation of GetByName returns an instance of ProgrammerLookupResult. If the lookup succeeded, it will be an instance of Programmer. Otherwise it will be an instance of LookupFailed.

The first thing to note is that TypeScript will not allow the client code to access the Programmer specific methods, like GetFavoriteLanguage until the type has been checked with a Type Guard. On line 21 of this example, the function checks the type of result. If this evaluates to true, the code on line 22 can call GetFavoriteLanguage without getting a compiler error.

This solves the two potential problems posed by the implementation with throwing Errors. The client code must check the type of the result, otherwise he code cannot compile. This results in fewer unhandled errors. Secondly, because it is easy to look at the definition of the union, it is clear to the developer writing the client code exactly which kinds of errors can be returned.

It is worth noting that the union type could be defined in line with the function. This would work identically to the previous example and it would make the code slightly shorter. I prefer to have the union type defined explicitly since in makes the intent of the union type clearer.

export class ProgrammerRepository{
    public GetByName(name : String) : Programmer | LookupFailed {
        // stub for more interesting data retrieval
        return new LookupFailed("Entity not found");
    }
}

Conclusion

Using Union Types to return errors from functions makes it clear to their clients what errors might be encountered. It guides them in how the potential errors must be handled. The result is more reliable software with fewer unhandled errors.

To view all code used in this post, please visit the GitHub repository.

The code changes in Roslyn between 7 and 8.

2018-12-18T04:00:00+00:00

For the second year in a row, Matt Groves is coordinating the C# Advent Calendar. It is a fantastic collection of posts on a wide variety of C# topics. I strongly encourage you to check out the other posts in series. For my entry, I wanted to take a look at the C# codebase behind the C# compiler.

I’ve been fascinated with the Roslyn codebase ever since it was announced. With C# 8 now released in a preview state, I was curious to know how much C# code it took to update the compiler from C# 7.0 to 8.0.

I cloned the Roslyn repository to my local machine. Then I used NDepend to compare the current state of the master branch (as of 12/17/18) to the branch named ‘dev15.0.x’. This is the branch that shipped with Visual Studio 15.0, better known as Visual Studio 2017. This should be the first released version of C# 7.0. It will be the baseline for my comparison. For this exercise, I wanted to focus on the compiler itself, so I only looked at three projects:

CSC.exe - the command line shell for the compiler
Microsoft.CodeAnalysis.dll - the core of the compiler
Microsoft.CodeAnalysis.CSharp.dll - the C# specific part of the compiler

This will admittedly leave out the changes that had to be made in the framework and additions to the unit test suite. But I am choosing to focus on the compiler.

NDepend reports that there are an additional 141,111 IL instructions. This represents 11.3% of the overall code base.

Looking at the changes at a more granular level we see there are 499 new types and 3,260 new methods. While some of the types seem to have come from refactoring, many of them are directly related to the new language features.

Here we can see some of the types that are directly related to new language features, like Ranges and functionality about nullability.

Here is the most impressive statistic that NDepend reports.

With all of the changes, the average method complexity basically remained flat. In most code bases, an 11% increase in functionality would lead to a large increase in complexity, if not technical debt. Here, the team has managed to add the new functionality without making the code messy.

There are some methods that got more complex out of necessity. For example, there is a method that returns the text of each kind of syntax item. With the additon of a few operators, it must get more complex.

Take Aways

Far too many code bases get messier and more complex as they evolve. But the C# compiler is a great example that proves that code can remain clean as it grows. The challenge here is that in your code base, you need to continuously work to keep the code clean and simple. Don’t settle for unnecessary technical debt and code rot as a code base ages. Work to keep in clean.

Azure Cognitive Services - Machine Learning the Easy Way!

2018-11-24T04:00:00+00:00

The power of machine learning algorithms has exploded in the last few years. Machine learning gives your app the ability to recognize objects in an image. Or it could detect the attitude of a user from something they wrote. This was beyond the capability of common apps until very recently.

But in order to use machine learning algorithms correctly, you complete three challenging tasks.

Find the right algorithm for your problem.
Collect and clean a sufficient amount of data to use as input.
Train a model.

While new tools have made these three chores easier in the last few years, the learning curve is still steep. Tools like Tensor Flow and ML.Net are very powerful and enable developers to create incredibly capable models. Investing the time in learning about machine learning would be very beneficial, but it would be a big investment.

But if you are a developer on a line of business application, you might not have time before your next deadline to learn all that you need to know. Wouldn’t it be nice if there was a way to have access to the power of machine learning without all the work of setting it up? That is the beauty of Azure Cognitive Services.

You can think of Cognitive Services as a friendly super intelligent robot that can answer questions for you. Let’s pretend his name is Coggy. You can hand him an image and he can tell you what is it in. You can hand him a block of text and he can tell you how the author felt when they wrote it.

The great thing about Coggy is that he has already been educated. You don’t have to teach him additional facts. Coggy has already learned the techniques he needs to know in order to find answers for you. He has already been trained to do the work that you need him to do. He is ready and able to answer your questions. And he is fluent in a language that you already know. So it is easy for you to talk to him. And Coggy does his own self-maintenance, so you don’t need to worry about keeping him running.

Azure Cognitive Services is a cloud-based service that provides access to machine learning models to perform many of the most common operations you would want in your application. Microsoft has already done the tedious work of collecting enough data and training a model. As a developer, all you have to do is make an API call to the service.

One great thing about Cognitive Services is that the API is a simple REST endpoint. So it is accessible from most common application types. Also, Microsoft provides wrappers for the API in JavaScript and C#, which you would expect. What is surprising is that they also provide API wrappers in other popular languages like Python and Java, making it easy for apps on all kinds of platforms to use for the power of Cognitive Services.

Cognitive Services has prebuilt models for 5 categories of operations. The Vision API analyzes images that are uploaded to the service. The Speech API provides text to speech, speech to text, and translation services. The Language API analysis on text, including sentiment analysis and content moderation. The Knowledge API enables you to build sophisticated question and answer behavior. The Search API gives you programmatic access to all of the features in the Bing search engine.

Now you can include these power operations in your next application. It will be as easy as asking Coggy a question.

Tuples in Visual Basic

2018-11-14T04:00:00+00:00

The new tuple syntax that was added to C# in C#7 has made my code cleaner and more expressive. It isn’t a feature that I use every day. But when there are multiple values that need to be moved as a cohesive group, it is very nice. And it is a big upgrade over the older generic tuple that always had its members named Item1 and Item2. While the generic tuple was useful for creating lightweight groupings, it hurt the readability of the code when the items had to be referenced. The new tuples can have custom names, meaning you get the benefits of tuples without sacrificing readability.

While I spend most of my professional development time in C#, I maintain multiple apps written in VB.Net. While the VB team has made a strategic decision to make their language more stable, they did add support for tuples in the most recent version. I used VB tuples for the first time recently and I am impressed with how clean the syntax is.

Declaring a Tuple

To declare a tuple, all you need to do is wrap two or more values in parentheses. In the following example, the code on line 2 is creating a tuple containing 42 and 84. Using this syntax, the items are still named “Item1” and “Item2”.

Function UseATuple() As Integer
    Dim pair = (42, 84)

    Return pair.Item1
End Function

The preferred way to declare tuples requires a bit more syntax, but it allows you to specify the names of the members. Using the colon equals syntax, line 2 is declaring a tuple with the same values as above, by now the items can be referenced with their identifiers, Magnitude and Direction. Line 4 uses one of the identifiers. The piece that you can’t see in a code sample is that the Visual Studio Intellisense is aware of the identifiers and will list them as autocompletion options.

Function UseANamedTuple() As Integer
    Dim vector = (Magnitude:=42, Direction:=84)

    Return vector.Magnitude
End Function

Passing Tuples

The most common use case for tuples is returning multiple values from a function. Without Tuples, you would have to decide between returning one of the values as an out parameter or creating a small data structure to contain them. To return a tuple for a function in VB, all you have to do is specify the members of the tuple in the method signature, again wrapping them in parentheses. The names are actually optional, but I would strongly recommend giving each member a name.

1
2
3

Function ReturnATuple(value As Integer) As (passed As Boolean, result As Integer)
    Return (True, value)
End Function

If you need to pass a tuple to a function, the syntax looks similar to the other cases. The tuple declaration is in a parameter. The only difference is that you do have to specify an identifier for the tuple.

Function PassATuple(pair As (isValid As Boolean, value As Integer)) As Integer
    If pair.isValid Then
        Return pair.value
    Else
        Return 0
    End If
End Function

Filtering Reddit Links with Azure Cognitive Services Sentiment Analysis

2018-06-18T04:00:00+00:00

One of the fascinating capabilities of Azure Cognitive Services is the ability to get a sentiment score for a text string. The score is a numeric value between 0 and 1. Values closer to 1 indicate that the statement is conveying a positive sentiment. Values closer to 0 indicate a negative sentiment.

This could be used to gauge how people feel about a product launch by getting the sentiment score of tweets that mention the product. On a recent .Net Rocks episode, Phil Haack talked about the potential to use sentiment analysis to monitor how GitHub contributors were feeling by analyzing the sentiment of their pull requests or their issues.

I thought it would be an interesting experiment to see if I could filter posts in a subreddit down to the posts that contain either a positive or negative sentiment. For example, could I get a view of the /r/programming subreddit that only contained posts with a positive sentiment? If so, it could be used to see what a particular subreddit is excited about. Conversely, it could also be used to filter a subreddit down to the negative posts.

The Technology Stack

In the past I’ve blogged about a library I built called Fognitive Services, that wraps the Cognitive Services API calls in F# functions. I’ve found that useful in the past and I decided to that NuGet package here.

Next, I had to figure out how to get data for a given subreddit. Reddit has a very nice and well-documented REST API. If possible, I wanted to consume that data with the FSharp JSON Type Provider. As a bonus, the latest beta versions of the package support .Net Core. So I would have a chance to test drive that new functionality.

To serve up the data, I decided to create an ASP.Net MVC Core 2.1 web application. The 2.1 version was released in the last few weeks and I wanted to try it out as well. Since of the focus of the first version of this app was connecting Reddit to Cognitive Services, I didn’t spend a lot of time making the UI look polished. I decided to simply generate the UI with Razor right in the web app.

Building the Back End with F#

F# Type Providers are almost magic. They are one of the best features of the language. Type Providers are plugins to the compiler itself that can generate strongly typed structures based on a data source. In this case, I want to generate code to consume the Reddit API.

The magic happens on line 4. I manually called the Reddit API and generated a file that contains the JSON response. The type provider is using that sample data to generate types for the Reddit API. After that single type definition statement, the RedditProvider type can now return strongly typed information from the Reddit API, complete with full Intellisense.

I could point the Type Provider directly at the Reddit API. But the Type Provider runs at compile time and I didn’t want my compile time to always include a REST request.

The file contains some additional functions I used to test out the Type Provider. But the code listed above is the entirety of what I needed to write to consume the Reddit API. The GetTitles function returns a list of stories for the given subreddit.

Before I could use Cognitive Services I had to create the resource in the Azure portal. I have some unused MSDN Azure credits that I am using for this experiment, but there is also a free trial that I could have used.

The next F# module takes the list of stories and transforms it into a list of inputs for Fognitive Services. Fognitive Services sends the inputs to Cognitive Services and returns a list of sentiments scores. I then use the List.zip3 function to combine the list of Reddit items, the list of inputs, and the list of sentiment scores back into a single list. Then I map the result into a list of RedditTitleSentiment so that the names are clearer.

Lines 5 and 6 are all that is required to get the sentiment score for each story. That is the magic of Azure Cognitive Services. It just a small amount of code you can have access to a very sophisticated machine learning algorithm.

In under 50 lines of code, I had everything I needed to retrieve the list of stories in a subreddit and assign them a sentiment score.

Building the Web Application

The web application is built with ASP.NET MVC Core 2.1. I admit I wanted to build something with the latest and greatest version of the platform and it went very smoothly. My primary concern was that there would be an issue integrating the C# web application with the F# class library. But I didn’t have any issues with that. The F# list type isn’t the same as the C# generic list type. But LINQ will work with both, so it is simple enough to convert the F# list type and an IEnumberable.

Using attribute routing, I added a few extra routes in the controller to make sure that the site would work with or without the subreddit name in the URL. I also choose to follow the Reddit pattern of putting “/r” in the subreddit URLS. This means that you can get directly to the CSharp subreddit by browsing to http://emotionalreddit.azurewebsites.net/r/csharp.

Hosting the Web Application

I created a new App Service in Azure to host the web application. I put it in the same resource group with the Cognitive Service. I was able to publish directly from Visual Studio, which was only noteworthy because there was both a C# web application and an F# class library.

Results

With the app fully functional, I can now get a sense for how well Cognitive Services evaluates sentiment. It definitely produces the desired effect. Like almost anything with machine learning, it isn’t perfect. From my observations, it seems to work better on longer chunks of text. Filtering stories in /r/todayilearned which tend to have longer titles seems more effective than posts in r/programming which tend to have shorter titles.

You can try out the site for yourself at http://emotionalreddit.azurewebsites.net/. Just specify the subreddit you want to filter and the sentiment level you want to see.

If you want to see the code, it is available on GitHub at https://github.com/pottereric/EmotionalReddit.

Creating a bot to play NES games with C#

2018-04-25T04:00:00+00:00

I have many fond memories of playing the original NES as a kid. So when I found out that someone had created an NES emulator that had an API, I was thrilled. The emulator I’m talking about is called Nintaco. And in this post I’ll show you how to write a simple bot in C# that plays an NES game.

Nintaco is a full featured NES emulator written by a developer that goes by ‘zeroone’. You can use it to simply play games. It also has tools for more advanced play like creating tool-assisted speed runs or even playing multiplayer games over a network.

It is compiled to a jar file, so there is nothing to install. Just download the file from the Nintaco site, unzip it, and run it. You will need the ROM files for any game you want to play. But there is no additional setup. At this point you should be ready to play some sweet 8-bit games!

There are other NES emulators, but no other emulators that I have seen have an API. There is another zip file available on the Nintaco home page that contains sample client code in C, C#, Java, Lua, and Python. It also contains documentation for the API. With permission, I packaged the C# proxy code into a NuGet package, so that it is even faster to get started.

To get started, create a new C# Console Application. Add a reference to the NintacoProxy NuGet package.

To initiate communication with the Nintaco app, call the initRemoteAPI method on the ApiSource static class. You will need to add a using statement for the Nintaco namespace for this code to compile correctly.

        static void Main(string[] args)
        {
            ApiSource.initRemoteAPI("localhost", 9999);
            new Program().launch();
        }

The next thing you need to do is register listener methods. These get called when various events take place in the game. In this sample, I have a method named launch that I call from the Main method.

        public void launch()
        {
            api.AddFrameListener(renderFinished);
            api.AddActivateListener(apiEnabled);
            api.Run();
        }

It adds listener methods for when a frame is rendered and when the API is first enabled.

Pushing Buttons

The simplest way to write a bot is just to program a sequence of button presses. At first glance this looks simple because there is a very straightforward API method to press a button.

ApiSource.API.WriteGamepad(0, GamepadButtons.A, true);

This function simulates a user pressing the A button on the first controller. But the key is understanding when this function should get called. Remember that your code runs in response various events happening in the game. They key event happens when a frame is finished being rendered, which happens 60 times per second. The sample code above registered the renderFinished function as the event handler for the FrameListener event, so it will get called rapidly.

Calling the WriteGamepad function inside the renderFinished function will only set the state of the button for that frame (1/60th of a second). When the next frame is rendered, the button states are cleared. So you will want to set the button state in a number of consecutive frames. The number depends on how long you want to simulate pressing the button. If you want to push the button for 1 second, you need to set the button state in 60 frames in a row.

You cannot set the button state to pressed, use a Thread.Sleep(1000) to wait a second, and then clear the button state. The Sleep method would block the execution of the emulator. So what you need to do is let the emulator run and check the clock every time the renderFinished method is called. This is similar to how you should handle timed operations in Arduino programming. In Arduino development, if the code sleeps for 2 seconds, the board won’t get any inputs that occur in that time frame. Similarly with Nintaco, if the code sleeps for 2 seconds, the game will lock up for 2 seconds.

When you want to simulate pushing the A button for 2 seconds inside the renderFinished function, you must keep track of the time when the button push started. Then set the state of the button to true in every call to renderFinished that falls within the 2 second time range.

State Machines

So how would you program a sequence of button pushes. Because a button push needs to happen over many invocations of the renderFinished method, you cannot rely on the calling methods in a sequence inside of a simple loop. You need to maintain the state of the sequence between calls to renderFinished. This can be done with a simple finite state machine. You need to setup up a variable to maintain which state the program is in. Every time renderFinished is called, it checks the current state of the program. If it is time to move on to the next state, it preforms the state transition. The next time renderFinished is called it will perform the function of the new state.

The sample program I wrote is playing Bomberman. The very simple game play algorithm is to perform the following steps repeatedly.

Move right
Drop a bomb
Move left
Wait for the bomb to explode.

Each one of those steps is represented by a state in our finite state machine. To keep track of time, I am simply using the current frame count. Nintaco has an API that returns it.

int currentFrame = api.GetFrameCount();

The code starts in the MoveRight state. Every time renderFinished is called in this state, the right arrow is pushed. Then the code checks to see if the desired amount of time has passed. If it has, it records the current time and transitions to the DropABomb state. Because the bomb can be dropped in one frame, it just pushes the A button and moves to the MoveLeft state. The state machine pushes the Left button for a set amount of time, then transitions to the Wait state. The state machine says in the Wait state long enough for the bomb to explode. At the end of the Wait state, the state machine transitions back to the MoveRight state, effectively starting the loop over again. Here is the entire state machine. You can see the full application in the Bomberman Bot repo on GitHub.

private void renderFinished()
{
    int currentFrame = api.GetFrameCount();

    switch (state)
    {
	case MoveRight:
	    ApiSource.API.WriteGamepad(0, GamepadButtons.Right, true);
	    if (currentFrame - stateStartFrame > spaceTraversalTime * 4)
	    {
		state = DropABomb;
		stateStartFrame = currentFrame;
	    }
	    break;
	case DropABomb:
	    ApiSource.API.WriteGamepad(0, GamepadButtons.A, true);
	    state = MoveLeft;
	    stateStartFrame = currentFrame;
	    break;
	case MoveLeft:
	    ApiSource.API.WriteGamepad(0, GamepadButtons.Left, true);
	    if (currentFrame - stateStartFrame > spaceTraversalTime * 3)
	    {
		state = Wait;
		stateStartFrame = currentFrame;
	    }
	    break;
	case Wait:
	    if (currentFrame - stateStartFrame > spaceTraversalTime * 6)
	    {
		state = MoveRight;
		stateStartFrame = currentFrame;
	    }
	    break;
    }
}

Running the Bot

To run your bot, launch the Nintaco jar. Open the ROM you want to play. Get the game to the state where your bot should start. In the case of my example program, I need to get past the start screen and into the game. The select Start Program Server from the Tools menu.

Click Start Server on the dialog that appears. Now you are ready to run your code. Launch your app, either by itself or in the debugger. Watch your little bot play the game.

From here you could start to build up your state machine to do more complex sequences. No matter how large the sequence, the pattern stays the same.

I hope you have as much fun with this as I do.

Analyzing the StirTrek conference sessions with Azure Cognitive Services

2018-04-08T04:00:00+00:00

In a previous post, I looked at how I could use Azure Congitive Services to look at the keywords from the StirTrek conference session titles. In this post I want to continue that exploration and see what we can learn about the service and the conference.

I am going to use the Text Analysis API. Specifically I am going to call the API that will return the key phrase or phrases for any input text. For example, if you pass it “Boom! From Combat Engineer to Software Engineer” (which is a real session at StirTrek this year), it will return “Combat Engineer” and “Software Engineer”. The API could also detect which language the given text was in. But since all of the StirTrek sessions are in English, it would not be very interesting in this experiment. There is also an API that returns a score representing the sentiment of the input text. I plan to explore that in a future blog post.

Experiment 1: Repeated Phrases in Titles

For the first part of the experiment, I wanted to see which key phrases appear in session titles both this year and last year. The StirTrek web site lists information for both years, so I can collect it with Canopy rather easily. As I demonstrated in the previous post, I used the Fognitive Services library to pipe the titles into Azure Cognitive Services. I used the Set operations in F# to create sets from the 2017 and 2018 title key phrases and then created a union of the results, giving the key phrases that appeared both years.

Using this method, I found that the key phrases that were repeated were ‘machine learning’ and ‘sql server execution plans’. One thing I noticed right away was that certain technologies were missing from the list. For example, ‘Angular’, ‘React’, and ‘UX’ appear in session titles both years. So Cognitive Services either has a hard time recognizing technology names or it didn’t see them as the key phrases in their respective sentences. The fact that the technologies don’t show up in the results means that we can learn something about the conference beyond what technologies are popular at the time.

The fact that machine learning showed up both years is interesting to me, especially since Cognitive Services is an nice abstraction layer over very sophisticated machine learning algorithms. For what it is worth, here are the machine learning sessions from both years:

Machine Learning in R (2018)
Using EEG and Machine Learning to Perform Lie Detection (2017)

While I was able to learn something from just looking at the titles, I’m going to run another experiment to see what I can learn by analyzing the session abstracts.

Experiment 2: Most Common Phrases in Session Details

For the second experiment, I wanted to look at the abstracts. We can still retrieve them with Canopy. But because there is so much more text in the abstracts that it isn’t helpful to look at which phrases appear in both years. There would be too many results. So instead I will append the key phrase lists together and count how many total times the phrase appears. You can see the raw results here. I think it shows something interesting about the conference.

The top two results (session, talk) are meta information about the conference, so that isn’t all that interesting. The next highest result is code, appearing 13 times. This is also uninteresting as it restates the fact that this is a developer conference. The same is true of developers appearing 9 times. The phrase way appears 10 times and ways appears 7 times. This would seam to indicate that the conference has an emphasis on showing how things can be done.

The first surprising result is that time shows up 9 times. There is clearly an emphasis on the fact that time is important to developers. The word tools appears 8 times, indicating that the conference wants to teach developers about the tools that can help them and presumably help them save time.

The result in the top 10 that surprised me the most was people, appearing 7 times. The related terms customers and teams both appear 4 times. Clearly the conference organizers recognize that some of the biggest challenges that we face as developers are not technical at all, but instead have to do with interacting with our fellow humans.

The interesting omission from the list is anything having to do with the movie theme. The conference takes place in a movie theater and the attendees get to watch the theme movie for the year when the conference is done. For example, this year all of the attendees will get to watch Infinity War at the end of the day. Much of the marketing uses this movie tie in. But looking at the themes of the titles and the abstracts shows that there is a ton of substance in this conference.

Next Steps

I was impressed with how easy it was to generate the results sets. It would be interesting to look at the same information but for a conference where I could get the results for multiple years. It would also be interesting to compare results between two conferences. I’ll try and track that down for a future blog post.

If you want to see the code that I wrote for the experiments, it is available on my GitHub page in the StirTrekKeywordAnalyzer Repo

Do you have an idea for something I should try to abstract from a conference data set with Cognitive Services? Leave a comment below with any suggestions. Or just find me on Twitter and let me know what I should be looking for.

T4: C#'s Little Known Code Generator

2018-04-05T04:00:00+00:00

The thing that I want to talk about in this post is not something new. It is something that has been available to Visual Studio developers for a few years now. It is not something that I see many people using. But it is something that I find to be very useful. I am talking about T4 templates.

T4 templates allow you to generate text in a similar way to how Razor pages allow you to generate HTML. It is a markup language that mixes C# with text. The text that is generated could be C# code or JSON. It could also be any text that you need to generate from some other data source.

Entity Framework made heavy use of T4 templates back in the Model-First paradigm. It would use the XML model of the database as the source and it would generate C# code with the T4 templates. If you wanted to alter the code that EF generated from the model, you could alter the T4 files.

This power is available to you for your code generation needs. You need some data source, which could be anything that is available to you with C#. As long as there is an algorithmic way to go from the data source to the output, you can generate it with T4.

It is simple to get started with T4. In an existing C# project, use the Visual Studio tools to add a new item.

On the Add New Item dialog you can search for T4 and you will see at least 2 templates available. I have the Tangible T4 Visual Studio Extension installed, so there are 4 additional templates.

Text Template

The Text Template is an item that will generate output right in your Visual Studio project every time the project is built. Effectively, these templates get transformed at compile time. They are generally used to generate code that will be compiled into the current project. This is the model that Entity Framework used.

Runtime Text Template

When you create an item with the Runtime Text Template, you create a class that can transform text when the program is executed. It is designed for scenarios where the output will live outside the project.

I build this blog with Jekyll, a static site generator. Every time I start a new post I have to create a new markdown file that follows certain conventions. I created a small C# project that uses a Runtime Text Template that takes some metadata and creates a markdown file with much of the necessary information already filled in.

In the past I have also Runtime Text Templates to read data from a database and generate C# static mock data for use in unit tests.

Software development involves manipulating a lot of text files. Anytime some of that can be automated means that you can get to your high-value work faster. If you haven’t explored T4 Templates, it is something I recommend you add to your toolbox.

Tutorial for working with Text Templates.

Tutorial for working with Runtime Text Templates.

Fognitive Services: F# and Azure Cognitive Services

2018-03-22T04:30:00+00:00

Machine Learning is revolutionizing how we think about processing data. But machine learning algorithms can have a steep learning curve. Luckily, Microsoft has released an easy way to access very powerful machine learning models through Azure Cognitive Services. They expose the power of machine learning through the ease of calling a REST API.

Microsoft provides good samples for calling into the API in C#, Java, JavaScript, PHP, Python, and Ruby. But I wanted to try it from F#. At the MVP Summit, I talked with Reed Copsey about this idea and he suggested I that I create an F# wrapper for the API. The wrapper could allow someone to call into Cognitive Services using idiomatic F# code. Thus the idea for Fognitive Services was born.

With help from Reed and Ryan Riley, I created the start of this project. It is built on the existing NuGet package for Cognitive Services .NET SDK. We started with the C# sample code and did a basic translation of it into F#. Then we refactored it to be idiomatic F# code. For example, the API now takes an F# list of tuples instead of a C# list of a custom data structure. This means that the API integrates smoothly with the other F# language tools like piping and pattern matching. Ryan and I recorded a screencast while we did part of this refactoring.

The GitHub repo contains some sample apps for each API. But I also put together a slightly more interesting demo project. The Cognitive Services Text API has the ability to pick out key phrases from a block of text. There is a wrapper in Fognitive Services for this in FognitiveServices.Text and there is a NuGet package for it. I thought it would be interesting to use Canopy to screenscrape the session list from the StirTrek conference page and pick out the key phrases from each session. This turned out to be incredibly easy. The entire project is available on GitHub. Here is the meat of the code.

All of the real work is done in the analyzeStirTrekTags function. Lines 9 and 10 load up the page in the browser. Line 12 initializes the Cognitive Services client. Lines 15 and 16 scrape the session titles off of the web page. The list of titles is seamlessly piped into the FognitiveServices API call on line 17 which returns the results. Line 19 prints the results.

This easy integration between two F# libraries was one of the main goals behind Fognitivie services. It combines the ease and power of Cognitive Services with the power and elegance of F#.

Making Canopy Tests Repeatable With Respawn

2018-01-27T04:30:00+00:00

Creating integration tests is a great way to validate the functionality of a web application, from the JavaScript all the way to the database. One of the challenges comes from the fact that data is being persisted to the database. In some cases, this prevents the tests from working properly when the test run is repeated.

As an example, let’s say that we have a test that creates a user account with a unique email address. The first time we run the test suite, the test will pass because the email address is unique. The second time we run the test suite the test will fail because the account can’t be created a second time. The test would fail even though the software was working correctly. False failures are always a headache with integration tests. So we want tools to solve this problem.

Recently, a library named Respawn was released for situations just like this. It was created by Jimmy Bogard, who is also the creator of popular libraries like AutoMapper and MediatR. Respawn will reset your database back to a known base state. It doesn’t this by intelligently deleting data out of your database. It is fast, so you can reset your database multiple times within a test suite. You can read the full description here. In this post, I specifically want to explore how Respawn works with Canopy.

Using Canopy and Respawn Together

Respawn was written with test runners like XUnit and MSTest in mind. But I believe it will work well with Canopy. Respawn and Canopy are both .Net assemblies, so they can be seamlessly integrated into the same project. I assume that Respawn could also be used with other .Net based web integration test frameworks, but I am primarily interested in Canopy.

I’m a huge fan of Canopy. I think it is a beautiful way to create automated integration tests for web applications. The fact that you get to write the tests in F# makes it even better. But it doesn’t have a built in way to reset your database back to a known state. Luckily, it is easy to use Respawn from within your Canopy test suite.

Like Canopy, Respawn is a NuGet package. So it is trivial to include it in your project. Once it is included, you just need to do two things:

Declare which tables in your database you don’t want to delete.
Determine where in your test suite you want to reset the database.

Declaring the tables that shouldn’t be reset

To start using Respawn, you need to create an instance of a Checkpoint. The Checkpoint object has a property name TablesToIgnore that tells the checkpoint which tables shouldn’t be deleted. All other tables will have their data deleted. Respawn will use the foreign keys to figure out which tables need to be cleared first. The property is an array of strings. Assigning it in F# looks like this:

For this example, I created a project that performs a simple test on the open source CrisisCheckin project. There are a number of tables that need to be populated in order for the application to work properly. So I included them in the list of tables to ignore.

Resetting the database

With the checkpoint created, you just need to call it’s Reset method to set the database back to the base state. The Reset method takes a connection string as a parameter. It relies on the fact that the integration test application can connect to the database and delete data from it. So if your integration tests run against a staging server where you don’t have direct access to the database, this solution won’t work for you.

In Canopy, we have methods that get called at known times that are great places to reset the data. The “once” function gets called one time at the beginning of each test context in your test suite. It is a useful place to call Reset.

If you want to reset the database right before every test in a test context, you could you use the “before” function. In this scenario, the quickness of Respawn would be very useful. If you wanted to reset the data after each test or at the end of a test context, you could use Canopy’s “after” or “lastly” functions.

Conclusion

I put together an exploratory project using Canopy and Respawn together. You can see it here. If you want to run it, you will need to clone the CrisisCheckin repo and run it locally. I was pleased with how smoothly the two libraries worked together. I was also pleased with how easy it was to use them to create integration tests that are repeatable and reliable.

Update

Chris Holt, the author of Canopy, gave me some great feedback. He pointed out that while cleaning the database makes some tests cases easier to write, there are also benefits to having a test suite that accumulates data. Tests suites that accumulate data can help you identify performance issues such as missing database indexes. If you want to have a large amount of data and still reset the database before a test run, you could use a tool that generates a massive amount of test data.

He also pointed out that you couldn’t use a tool like Respawn in a scenario where multiple people are testing against the same database because Respawn would nuke everyone’s test data, not just yours.

Test Drive- Serverless Web Applications

2017-12-30T14:30:00+00:00

The last time I was in the market to buy a car, I started by reading a lot of reviews. I talked to people that already owned the cars that I was interested in. I did comparisons of my top choices online. But before I would ever buy a car, I took it for a test drive. In fact I took several cars for test drives before buying the one that I liked the best.

I often take a similar approach with technology. I frequently listen to podcasts or check sites like Hacker News just to know what the trends are. I follow a large number of people I respect on Twitter so I can know what new tech they are excited about. At conferences, I’ll start up random conversations with people in the hall or in line for meals in order to know what they are interested in. When a technology seems interesting enough, I’ll start to read up on it, reading some reviews, overviews and blog posts (just like you are right now). But before I would ever use some new technology on a production software product, I take it for a sort of test drive.

The technology trend that is getting hard to ignore this year is serverless web applications. You can argue that the name “serverless” is misleading, but I’ll use it because that seems to be what people are calling it. In serverless applications, the server is obviously there. But the technology tries to abstract it away from the developer’s concern. Because I’ve done much of my work on the Microsoft stack, I decided to try out the Azure serverless offerings.

The Application

For it to be a proper test drive, I wanted to come up with a nontrivial application. I decided to build something that would monitor my twitter feed for URLs that were tweeted by multiple people. So these were the requirements that I came up with:

Monitor my Twitter feed for tweets containing URLs, and store the tweets that are found.
Expand shortened URLs to the target URL.
Query the collection of full URLs for links that appear multiple times.
Display the results on a web page.

It isn’t the worlds greatest application. But then again, no test drive is the world’s greatest road trip. I figured that there was enough in these requirements to get a good feel for how these new technologies worked in real world scenarios.

The Architecture

In order to monitor my Twitter feed, I used Microsoft Flow. It has a WYSIWYG editor that makes it dead simple to create simple workflows. It has adapters for Twitter as well as for CosmosDB, which is what I chose for my storage needs. CosmosDB is the new NoSQL offering in Azure. It works particularly well in serverless applications, so I wanted to kick it’s tires. I also wanted to try out the ability to write stored procedures in JavaScript. So I did part of the querying in a stored procedure.

I wanted to try out Azure Functions to handle the business logic. I use a timer to kick off a function that expands the URLs. I use another function to aggregate the data and serve it up as a REST API. That API is consumed by an HTML page that has no server-side rendering. There is an Azure Web App that hosts the HTML, but the page only uses Axios to call the REST API and Vue to display the data.

Here is the whiteboard diagram of the system.

The Result

In the end, I web page I can load that will display the desired results. It isn’t the next Uber of anything. But it does meet the requirements and it does qualify as ‘serverless’. Here are my opinions of the technologies that I tried.

I detailed my thoughts on Microsoft Flow in a previous blog post. The quick recap is that it was delightful to work with. The UI was intuitive. I got the functionality that I wanted very quickly.

I really enjoyed working with Azure Functions. I’ll have a longer blog post later with more details. But here I will just say that they functioned the way that I expected and I was able to focus on my logic, not on the hosting aspects. So it succeeded in abstracting away the server concerns.

CosmosDB was fascinating. Of the things I tried in this project, it had the steepest learning curve. But it may also have the biggest benefits. I’ll also put a much longer review in a future blog post. For the time being, I’ll simply say that CosmosDB has a bright future. But right now it is a still a very young project, which leads to pain points at times.

The web page was so trivial that I would consider it a proper test drive. But I was impressed how quickly I was able to put something together with Vue and Axios. For bonus points, I wrote the web page on an Ubuntu machine using only Vim. I deployed it to Azure with Git. But even with these arbitrary constraints, the whole development process was quick and easy.

While the whole system isn’t getting used heavily, I’ve been impressed with how little it is costing me. The timers run parts of the system every day. I have been loading the web page almost every day. The whole thing is costing me less than $60 a month, which is easily within my $150 monthly allowance with my MSDN subscription.

Overall, I was very impressed with how well the various peices fit together. The integration between Flow and CosmosDB worked great, especially considering it is still preview functionality. The integration between CosmosDB and the Azure Functions was also very good.

Conclusion

I can see why there is so much hype around serverless applications. It was nice to be able to write my business logic and ~~never~~ rarely think about the web server. Having completed the test drive, I have already recommended Azure Functions for use in an upcoming project at Aptera. CosmosDB has a ton of promise. I’m sure I’ll use it professionally in the future as well. Microsoft Flow was great for my personal project. On a bigger project I would probably recommend Logic Apps.

It was a great learning experience building with these new technologies and I look forward to working with them in the future.

Visualizing Projects With NDepend

2017-11-18T04:30:00+00:00

As a consultant, I am frequently working in new code bases. Sometimes they are quite large and have existed for many years. In these cases, it is important to get a feel for how the project is structured. It is important to understand where the important parts of the code are. NDepend is a tool that helps developers understand what is going on in a code base. It provides graphical summaries of the structure of the code. In this blog post, I want to see what I can learn about a few projects, just by using NDepend. (In the interest of full disclosure, Patrick did give me a license. )

I want to start by analyzing a simple project. So I am going to look at the Nintaco proxy code. I opened the project in Visual Studio and attached NDepend to the project. I immediately got a report on the state of the project. It told me that there were 999 lines of code (in the green box below). It told me that it only has one dependency, .Net Standard.

What caught my eye was the 1 Quality Gate fail and the 2 Critical Rule violations. The Quality Gate failure is referring to the Rule violations. By clicking on the rule violations, I am immediately given more details on what rules failed and were the offending code is located. There is an impressive amount of detail about the error, where it is, how to fix it, and where to get more information.

There are a lot of metrics on the dashboard that can show how the quality of the code is trending over time. They seem really valuable. But because I have just attached NDepend to the project, those metrics don’t have much to show right now. I’ll have to revisit them in a future blog post.

One of the most useful tools in NDepend is the Code Metrics view. It shows each method in the module as a rectangle. The size and color are determined my metrics that you can select. By default, the size is determined the number of lines of code and the color is determined by the cyclomatic complexity. In the default view for the Nintaco Proxy, you can see that there are a huge number of methods that are approximately the same size and complexity. This makes sense because its chief responsibility is proxy method calls to a server. Each method call takes about the same amount of work to call the proxy.

But at a glance, you can see that there is one method, ProbeEvents, that is more complex than the others. This makes it a prime candidate for refactoring or for some unit tests. This is the beauty of NDepend, it makes valuable information about your code base available at a glance.

Analyzing a Larger Project

In order to exercise the dependency graph, I wanted to analyze a larger project. I chose to look at the Crisis Checkin project by Humanitarian Toolbox. This is what the Dependency Graph looks like:

With a quick glance at this image, you can start to see how the project is architected. It clearly has a traditional 3 tier architecture. You are also seeing something about the relative size of the different assemblies based on the size of the boxes. The line thickness is telling you how many members in one assembly depend on a member of the other assembly.

What you are not seeing is that the graph is totally configurable. We could make the shape sizes dependant Cyclomatic Complexity instead of lines of code. The other amazing thing is that everything in this graph is clickable. You can click on any item to get more information or to go to the code. Clickability and configurability are very common in all of the NDepend graphs.

Again the Code Metrics view provides an easy view of the complexity of the project.

Having the Right Map

Where I am from in Indiana, the only kind of map you would ever need is a roadmap. There is almost no terrain to speak of. So if you see from the roadmap that the trip is 60 miles, you know what kind of trip you are in for. But if you live in the mountains, this might not be the case. The distance only tells half the story. You also need to know about the terrain. In these cases, it is helpful to have a topographic map, one that tells you what kind of terrain you will be driving in.

I feel like NDepend gives you topographic maps of your projects. The solution explorer is like a roadmap that tells you where things are. The graphs in NDepend tell you much more about the terrain of the project. It can tell you which parts will be steep when you need to make a change.

There are a lot of other great features in NDepend that I haven’t covered here. I’ll come back to them in future blog posts. For now, I’m having a lot of fun using NDepend to explore my projects.

Test Drive => Cosmos DB with Microsoft Flow

2017-10-24T04:30:00+00:00

Microsoft recently a preview version of their Microsoft Flow connector for CosmosDB. If you haven’t tried out Flow before, connectors are the tools that allow you to integrate external services into your workflows. Connectors allow you to read or write from data sources like Twitter, Office 365, Dropbox and many more.

Flow already had connectors for SQL Server, PostgreSQL, and MySQL. They work great for situations where you want relational data. But I find that especially when I am working with disparate systems in Flow, it is much easier to store data in a less structured way. This makes working with CosmosDB and it’s document store a very natural fit.

To do the CosmosDB integration, I opened up a Flow. I already had a flow that would pick certain tweets off of my timeline, so I just edited that flow. I clicked on the “Add an action” button. From the dialog, I searched for “CosmosDB” and in this case, I picked “Azure Cosmos DB - Create or update document”. But as you can see, there are many other CosmosDB actions I could have taken.

Because it was the first time I had connected to CosmosDB, I was prompted for connection information about my Cosmos database. I had already created one inside the Azure Portal, so I just had to enter the information. Otherwise, I would have needed to create the database. The last thing I had to do was fill in the JSON that I wanted to insert. The Flow editor allows you to click on the data fields that you want to include. So in my case, I was able to include fields from my twitter input right in the JSON.

The only hitch I ran into while building the workflow was that the Flow engine didn’t require me to have an “id” field in my JSON document. The first time I ran the Flow, Cosmos returned an error because there was no “id”. Once I added it, my flow worked correctly. Becuase I was storing tweets, using the tweet id and the document id was a natural fit. I also had a small issue at first because I pasted in some JSON and the JSON editor didn’t think it was valid. Once I removed the unnecessary whitespace in the JSON everything was fine. Since this is still a preview version of the CosmosDB connector, I’m sure these little issues will be resolved before the final release.

Summary:

CosmosDB is a great way to persist data from Microsoft Flow
Make sure your JSON documents have an “id” field before storing them.

Net Standard Case Study

2017-10-19T14:30:00+00:00

Much has been written about how many APIs have been added to .Net Standard between version 1.1 and 2.0. I wanted to share a practical example of how this API growth has improved the product.

There is an NES emulator named Nintaco. It can be controlled via an API, meaning that you can write a bot to play an NES game. There are API examples in several languages including C#. I got permission from the author to bundle the C# API as a NuGet package. I wanted to make the package a .Net Standard class library so that could have the widest possible audience. But when I first tried to create the NuGet package, it couldn’t compile under .Net Standard, which at the time was a version 1.1.

To integrate with the Nintaco API, the emulator makes some sockets available and the API talks to them over TCPIP. Many of the networking APIs where missing from .Net Standard as of version 1.1. Here were the errors that I got:

The type or namespace name ‘Sockets’ does not exist in the namespace ‘System.Net’
‘BinaryWriter’ does not contain a definition for ‘Close’
‘BinaryReader’ does not contain a definition for ‘Close’
The type or namespace name ‘TcpClient’ could not be found
The type or namespace name ‘BufferedStream’ could not be found
The name ‘Thread’ does not exist in the current context

It makes sense that the networking and threading APIs would be missing initially because they are obviously coupled to the underlying operating system, making them more complex to write in a cross-platform environment. But this prevented me from being able created .Net Standard 1.1 library for the Nintaco API.

In .Net Standard 1.3, the Sockets class was added, resolving one of the errors.

In .Net Standard 1.5, the BufferedStream class was added, resolving another of the errors. This left 4 errors.

In .Net Standard 2.0, all of the remaining classes that I needed were added. So while the missing classes were a deal-breaker in .Net Standard 1.1, version 2.0 had absolutely everything I needed. The package is now published to the NuGet repository. I’ll have more about how to use the package in a future blog post.

Pattern Matching = Conditional and Assignment

2017-10-07T14:30:00+00:00

I’ve been in love with pattern matching in C# since I first heard it was going to be added to the language. I had used it in F# and was excited to use it is C#. In a conversation I had this week I realized something important that I had known about pattern matching, but I had never articulated it. One of the most powerful things about pattern matching is that it cleanly combines a conditional statement and an assignment statement into a single line of code.

Let’s look at the following code taken from the Entity Framework Core code base

On line 3 the code will switch on the expression variable. If the expression is an instance of ColumnExpression, the statement on line 5 will evaluate to true, but it will also assign the variable named columnExression. The variable named columnExpression will have a type of ColumnExpression and will be assigned the value stored in expression. Similar things happen on lines 7 and 9 if the type of expression is ColumnReferenceExpression or AliasExpression.

The magic here is that in a concise but readable statement, the type is evaluated and the variable is assigned. This makes the code much cleaner than if the type check and the expression were in separate statements for each possible type. Also noteworthy is that there is no need for a null check because a if expression is null, the switch statement will match on the default case. (If it were present, it would match on “case null:”.)

So the pattern matching is C# combines the functionality of “is” and “as” into one statement. It can be used directly in a “switch” statement. It resembles a more powerful version of TryParse in that it can both evaluate the type and do the assignment. It also resembles a regex in how it can determine the validity of the match and pick values out of the input. Of course, TryParse and regex only work on strings where pattern matching can work on any type.

Pattern matching combines aspects of all of these constructs into one powerful feature.

Learning From the Atari 2600

2017-07-08T14:30:00+00:00

The majority of the code I write completely abstracted away from the hardware it eventually runs on. If I am writing a Web API in C#, my code only knows what is will be run by the CLR, which may be running a server or an Azure host, but the code doesn’t need to know which one. And it certainly doesn’t need to know what kind of RAM is installed on the machine that it eventually runs on. It doesn’t necessarily need to be concerned with how much RAM the machine has because it can safely* assume that there will be enough. And when I write client side code, it is even further removed from the hardware.

Racing the Beam

I recently read ‘Racing the Beam’ and the most fascinating thing about it was that the code described in the book had to be intimately aware of the hardware in the Atari 2600**. It was a stark contrast to the code execution environments that web developers work in these days.

The title of the book, Racing the Beam, acknowledges the fact that the Ataris were built to run with old CRT TVs that worked by continuously painting the screen with an electron beam. Because the Atari was single threaded and it did not have a graphics processor, the code for the actual game logic could only run while the beam was moving back to the left to get ready to paint the next line or back to the top to paint the next frame. The rest of the time the processor was responsible for painting the screen.

“The programmer must carefully “cycle count” processor instructions so they execute at the right time.”

The fact that the developers had to count CPU cycles is incredible. What is just as astonishing is that the machine only had 128 bytes of RAM. Bytes. Not Gigabytes, or Megabytes, or Kilobytes. Bytes. As a point of comparison, my iPhone has 128 GB of storage.It literally has a billion times more capacity than the 2600.

When I was writing software for Palm OS, I had to support devices that only had 1 MB of RAM and that seems tiny. A web developer would do better being constrained to only using half a keyboard than trying to get anything done in 128 bytes. To be fair, the code was not loaded into memory but was instead executed directly from the cartridge ROM. But that is still a minuscule amount of memory.

Technical Wizardry

The cartridges didn’t have much ROM space either. These limitations forced the programmers to be very creative. In the popular game Yars’ Revenge, the developer wanted to place have a multicolored safe zone. But creating a bitmap for this safe zone would take up too much memory. So he loaded chunks of the code itself as the bitmap. So when you are playing the game, you are seeing the game’s code rendered as an image.

Yar’s Revenge has another technique that really blew my mind. The developer knew the opcode for a function return statement and he had a sprite that happened to start with that same hexadecimal code. So he laid out the ROM so that the first byte of the sprite could also serve and the return statement of the previous method.

“In this code, then, the value $60 serves two purposes, as the opcode RTS when it is encountered in program flow and as the value $60 (binary %01100000) when it is read as data. As with the rendering of the Yars’ Revenge neutral zone, this is an example of the use of the contents of ROM—only a single byte, in this case—as both code and data.”

One of the things that made this book great for me was that I remember playing these games. I remember them fondly as a form of entertainment. Now I also get to appreciate them as brilliant accomplishments in software development.

The book spends a lot of time talking about the great lengths that the developers had to go through to render graphics on the screen. It doesn’t spend as much time talking about the personalities involved, such as Nolan Bushnell. Those stories are covered in other books. This book stays very technical. It spends a lot of time talking about the capabilities and limitations of the hardware. I can’t do the book justice to try to summarize it here. But I would strongly recommend that you go read it for yourself.

Lessons Learned

The primary lesson that I learned from this book was about the value that a developer can get from a deep knowledge of their platform. Even though the platforms that I develop for have far more resources, knowing exactly what can be done enables clever solutions. Having a deep knowledge of the platform enables me to use it to it’s full potential.

Just as the game developers for the Atari 2600 had a deep knowledge of the hardware they were running on and a very detailed knowledge of the TVs they were using as display, I should have a deep knowledge of the web frameworks that I use and the web servers that run them. This kind of knowledge will help me build faster, better looking applications for my users.

Hat tip to Matt Groves. It was episode 16 of his Cross Cutting Concerns podcast that first introduced me to the book. Check out the episode for yourself.

Yes, I know this isn’t strictly true. But that is outside the scope of this post.

** The console was originally released as the Atari VCS. The book sticks with that name. I remember it from my childhood as the Atari 2600 and I can’t break the habit of using that name.

Pattern Matching on Types

2017-03-16T14:30:00+00:00

In the previous post, there was a subtle feature that I didn’t highlight. That was the fact that the switch statement was switching on a tuple, which is a struct. This might not seem like a big deal, but before C# 7.0, if you tried to switch on a struct, you would have gotten the following error message:

“A switch expression or case label must be a bool, char, string, integral, enum, or corresponding nullable type.”

What C# 7.0 gives us is the ability to switch on any type and match them with Type Patterns. It makes sense to match on classes when you consider that C# is an object oriented language. Where functional languages provide extra tools to match on lists (which are integral to functional programming patterns), C# gives us the ability to match on an object’s place in the inheritance hierarchy.

You could make an argument that the client code shouldn’t switch on the different types and that the different behaviors should be accessed through polymorphism. I would generally agree with that. But in many cases, you can’t alter the classes you are working with. For example, if you are working with UI objects in a WPF application, you can modify the classes.

So let’s look at a WPF example. Let’s say that you have a dialog that has multiple types of controls, and you need to be able to reset all of the controls back to their base state. The trick to this is that depending on the control, the reset behavior is different. We could do this in C# 6 with if statements, is checks and as casts.

Now let’s look at the same code with C# 7.0 Pattern Matching.

First off, this cuts the number of characters inside the foreach loop from 252 down to 172 and it cuts down the number of lines by 3. What is more important is that the code is much more readable.

Part of what makes this so much better is that the Pattern can declare a variable. So on line 10, if the control that is being matched is a ComboBox, a variable named cb will be created that is already a ComboBox. This removes the need for the is and the as before accessing the SelectedIndex property.

As I’ve said in the other posts in this series, this isn’t a massive new feature of the language. But I hope that you will agree that this is an incredibly useful feature that will improve your code. If you disagree, please let me know in the comments below. I’d love to hear your perspective.

Pattern Matching and Tuples

2017-03-15T14:30:00+00:00

As I discussed in my previous post, pattern matching in a new feature in C# 7.0 that provides a more sophisticated way to write selection statements. The situations where sophisticated selections statements more necessary is when selecting on multiple criteria. Often this is seen in other languages when matching against tuples. In an older post, I looked at how Swift uses tuples and pattern matching and used the FizzBuzz problem as a solution. Since C# 7.0 also introduced a better syntax for tuples, I thought would be helpful to revisit the solution, this time in C#.

In the code, the method IsMultipleOf3or5 returns a tuple made of two bools. In this instance, the members of the tuple are given names to make them more readable. This is a big improvement over the old C# tuples that had to be named Item1 and Item2.

The result of IsMultipleOf3or5 is the value that is the subject of the switch statement. Each case statement uses a pattern that names the variable that is being switched. In each case the variable name is status. The when clause then uses the property names defined in the tuple to filter the selection.

Like I said in the previous post, this logic could certainly be implemented with if statements, but the new pattern matching logic makes the code cleaner and easier to read.

Benefits of Pattern Matching

2017-03-05T14:30:00+00:00

As someone who dabbles in F#, I was excited when I heard that C# was getting Pattern Matching. In this post I want to look at why it is so beneficial.

The three basic control flow mechanism available in all programming languages are selection, iteration, and calling subroutines. In same way that a foreach loop is an evolutionary step for iteration, pattern matching is an evolutionary step for selection. It isn’t a ground breaking new paradigm like async/await or Linq, but it is something that will be useful at a fundamental level. The foreach loop doesn’t allow you to do things that you couldn’t do with a for loop or a while loop, it allows you to do them faster and with cleaner code. In the same way, pattern matching won’t allow you to do things you could do with if statements, it just enables you to write them better.

Here is an example from the Roslyn repository. As of this writing, you could find the source file on GitHub.

It is code that looks for possible refactorings where digit separators could be used. So it would recommend digit separators for decimal constants with more than 3 digits and digit separators for binary and hexadecimal constants with more than 4 digits.

The code switches on an enum that describes the type of numeric constant. Each case statement has a when clause that only allows selection of the current case when a length requirement is met. The combination of the enum type and the when clause make up the pattern.

This could all be done with if statements, but it would be more code. The benefit of using pattern matching in this is that it case makes the code cleaner and more readable.

Session Recommendations for CodeMash '17

2017-01-05T14:30:00+00:00

One again, the content at CodeMash is going to be fantastic. There are great sessions all over the place. If you are attending, you will be like a blind dog in a butcher shop. Even if you don’t know where to go, you will find good content. That be said, it can be hard to pick which sessions to attend. I’d like to offer some humble suggestions. But before I do that, I’d like to offer a few caveats.

Caveat 1: I am going to limit my suggestions to sessions that I have seen in some previous form. This means that I am skipping over a bunch of great sessions by great presenters because I haven’t seen the content before. It also means that I am not going to mention sessions by the big name presenters like Jon Skeet, Cory House, Seth Juarez, or Jenifer Marsman. Their sessions would be worth seeing, but you probably already know that.

Caveat 2: I spend most of my developer time in the .Net ecosystem so I am less aware of who the good presenters are or what the cool topics are in the other ecosystems like Ruby, Python, Node, etc. I’m sure there is a ton of great content in those tracks that is outside my radar.

With that being said here are some recommendations:

Language Geek Sessions

I really enjoy learning about programming languages and their nuances. So I really like Craig Stuntz’s talk titled Incredibly Strange Programming Languages. It is a fascinating look at some languages that you have probably never heard of. I also like Rachel Reese’s talked called A History of F#: From Euclid to Type Providers It focuses on the history of functional languages with a focus on F#. Both of these sessions will give you a better perspective on the languages you are already using.

If you haven’t looked at using a functional language before, you ought to check out Getting Started with Functional Programming in F# from Reid Evans. He has a great story about he was able to adopt F# in his professional work and how it benefited him.

ES6

It goes without saying that JavaScript is a big deal. Hence it is important to know where the language is going, especially because you can take advantage of it now. You can get the intro from Jeff Strauss in Adding ES6 to Your Developer Toolbox and take a deeper look in Deep Dive Into ES6 Generators with Jonathon Mills. Both talks will give you tools that you can start using immediately.

Soft Skills

While I am most interest in the technical side of our profession, I know that the nontechnical side is just as important and CodeMash has sessions to help in that area too. Cassandra Farris will teach you about career growth in Career Growth Questions You’re Afraid to Ask and Jay Harris will share about working in a team environment in Make Donuts Great Again: The Tale of the Broken Build.

.Net

If you are an ASP.Net developer, there is a good chance that you will need to deal with Authentication and Authorization in .Net Core. Ondrej Balas will help you understand how this all works in ASP.NET Core Identity Management.

Other

One of the greatest things about CodeMash is that it gives you a chance to learn about things in other tech stacks and you should absolutely check out some topics outside of what you are currently using. And if you are in one of those other communities please comment below about what sessions I should see or tell me I twitter (@pottereric). I would love to know what you think I should see.

The Rest of the Time

Lastly, make the most of the opportunities you have outside of the sessions. Talk to the other attendees. For many of us, this can seem like a daunting task. But Jeremy Clark has some great advice over on his site Becoming A Social Developer. If you don’t know what to say, just ask someone “What kind of code do you write?” There is a good chance they would love to tell you. Take the time to chat with people while you are in line for meals or while you are in the hall. Don’t always sit with people you already know. You will be surprised what you can learn from these interactions.

Databinding SVG with Aurelia

2017-01-01T14:30:00+00:00

Recently my son was looking at a globe and asking me questions like ‘Is Ecuador bigger than Texas?’ I found the answer quickly on Wikipedia, but it got me thinking that it may be fun to have a graphical representation of the comparison.

I decided to put the GUI together as a web application using Aurelia. I wanted to have a list of checkboxes to allow my son to select the countries or states that he wanted to compare. When geographic regions were selected, circles would be drawn that were proportional to the square miles in the region.

My first thought was to have a delegate on each checkbox that would draw the circles programmatically. That certainly would have worked, but then I stumbled upon this Stack Overflow answer from @AshleyMGrant.

It helped me realize that because SVG elements are regular HTML elements, you can bind to them with Aurelia just like you would bind to any other elements. All I had to do was use the repeat.for binding on a circle element. Each circle is bound to a geographic region in the view model, the same elements that were bound to the check boxes. Each geographic region in the view model has a number of square miles associated, and the radius of the circle is bound to this value.

One of the things that I like about this solution is that it takes advantage of the Model View Presenter/Controller pattern. The model contains all of the relevant data. It doesn’t concern itself with how it is displayed. The view has two different representations of the data, one as a list of checkboxes, one as graphical objects.

AttachTo and the Art of Doing One Thing Well

2016-12-01T14:30:00+00:00

That is a drywall saw. You can’t use it to measure a board. You can’t use it so see if something is level. You can use it to remove a screw. You can’t use it to drive in a nail, or at least not very efficiently. In fact, the only thing it does well is cutting drywall. It isn’t a tool you use frequently. I’ve only used mine a handful of times. But when you need it, which is generally when you are hanging drywall and you need to make a hole for an electrical fixture, it is incredibly useful and effective.

Some Visual Studio extensions, such as Viasfora, are like screwdrivers. You use them all time on almost any kind of project. Extensions like CodeMaid are like a multi-tool. They have a large number of things they can do, so you end of using them very frequently. Some extensions are like drywall saws. They do one thing and they do it well.

AttachTo falls into this last category. The only thing it does is that it gives you menu options to attach to IIS, IIS Express, or NUnit. That is it.

One of the situations that I work in on a regular basis is developing a class library that is used by a web application. So when I need to debug it, I need to attach to IIS or IIS Express. This can be done without the extension by clicking the ‘Attach to Process’ menu option and then selecting the processes for IIS. You will then have to confirm that you want to attach to these processes. When you have to do this repeatedly, it is much nicer to have a single menu option.

If you aren’t working with drywall, you don’t need a drywall saw. If you don’t need to Attach to an IIS instance, don’t install this plugin. But if you are like me, and you need it, you are going to love this thing.

What I would encourage you to do is browse the Visual Studio Extension gallery for tools that you would use. There are tons of tools out there and almost certainly one of them will make your development easier or faster. There very well maybe something that you do on a regular basis that could be automated or expedited. That is the beauty of the Visual Studio and Visual Studio Code Extension ecosystems. You can add features to meet you specific development needs. Some extensions have tons of features. Some have features you will use all the time. Some have one feature that you use occasionally. All of these kinds of extension make you development life better.

Clean Code Faster with CodeMaid

2016-11-30T14:30:00+00:00

If you have been writing software for any length of time, you have almost certainly found yourself staring at a source file where you have made a mess. It had been nice and organized, but then you needed to add a feature. you wrote your tests and implemented the code. Your application is running as fast and smooth as a BMW on the Autobahn. But all of your changes have left the code as disheveled as the hair of a teenager just getting out of bed.

You need to go back through and format the code, remove the extra blank lines, and so on. This would be time-consuming to do by hand. This is where the Visual Studio extension named CodeMaid comes in. It removes unnecessary blank lines, removes end of line whitespace, runs the Visual Studio formatter, and other things that improve your code.

It also gives you a new visualization of the source file in the CodeMaid Spade tool. One of the nicest features of CodeMaid is that this tool allows you to drag a drop members within the source file. This allows you to quickly arrange the methods, properties, and fields into an order that makes sense.

In this gif, you can see that by dragging and dropping a property in CodeMaid Spade, the property is moved in the code file. This works for functions as well.

In the interest of full disclosure, I should say that the author of this tool, codecadwallader, is a long time friend of mine. That certainly makes me look at the tool through rose colored glasses. However, I have been using this tool ever since it was created about 10 years ago and I have benefited from it time and time again.

There are a large number of other things CodeMaid can do to clean your code. You can see the on the CodeMaid site. But the greatness of CodeMaid doesn’t come from it’s large number of features, it is that it does one thing really well, that is it cleans your code. And that is something that we all end up doing on a regular basis.

Improve Code Readability with Viasfora

2016-11-14T14:30:00+00:00

I can remember the first time I wrote code in an editor with syntax highlighting. It seemed to make the code so much more readable. Instead of looking at a wall of black and white text, I was getting visual cues about what was going on. The thing that I love about the Viasfora extension for Visual Studio is that it takes this concept to the next level.

Viasfora simply adds color to source files to give you more clues about what is happening. The color is subtle. It doesn’t make you code look like a Jackson Pollock painting. It simply gives you more glanceable information about the structure of the code.

Here is a comparison of what a chunk of C# code looks like with Viasfora (on the left) and without it.

The first thing you notice is that the access modifier is gray. So in this case, “public” is not the same blue as the rest of the keywords.

Secondly, flow-control keywords are displayed in red-orange, making them stand out. In this case, “foreach” gets your attention right away.

Thirdly, all of the braces and brackets are colorized so that matching pairs also match in color. This is featured is called ‘Rainbow braces’. In this example, the purple braces match each other. The orange parentheses match the orange parentheses and the red parentheses match each other. In my mind, this is the killer feature of Viasfora. It makes it much easier to quickly see where the block begins and ends. This is especially when you have multiple closing parentheses in a row. It is quickly obvious where parentheses are missing as well.

Viasfora also does a good job of being configurable, all of the features I mentioned can be turned on and off. Each of the colors can be configured. It also supports a range of different languages, including JavaScript and F#.

There are other features that I haven’t mention that you can read about on Visfora.com. You can get it on their page in the Visual Studio Gallery or through the Extensions and Updates menu inside Visual Studio.

I highly recommend this extension, it gives you additional functionality without ever getting in your way or slowing you down.

Visual Studio Extensions

2016-11-10T14:30:00+00:00

As a developer who primarily creates web applications on the .Net stack, I spend a lot of time in Visual Studio. For smaller edits, I’ll use tools like Visual Studio Code of gVim. But most of the time I want to be able to take full advantage of the full array of tools Visual Studio provides. Whether I am creating projects, editing files, or debugging code, Visual Studio offers an unmatched feature set. But as many features as it offers, it doesn’t have everything I want. That is where extensions come in.

Over the last few days, I’ve been thinking about the role of extensions. As I see it, there are two primary benefits of extensions. The first is to add functionality that Visual Studio doesn’t have yet, but someday it should. The other benefit is to offer features that only apply to a subset of users.

Technology changes very rapidly. When Visual Studio 2015 originally shipped, WebPack didn’t have nearly as large of a user base as it has today. So it didn’t make sense to support it out of the box. But now it is much more widely used. Instead of waiting for another Visual Studio release, we can have WebPack support now via the WebPack extension. It is possible that the next version of Visual Studio will include this functionality and the extension won’t be needed. But until then, we have improved functionality with the extension installed.

There are other features that will never make it into Visual Studio because a large number of users wouldn’t care about them or they would be opposed to them. The quintessential example of this is VsVim. It provides Vim keybindings in Visual Studio. I can’t live without this extension. I installed as soon as I install Visual Studio. However, a large percentage of developers would hate the idea of having to use Vim commands to work in Visual Studio. So this is a case where an extension benefits a subset of users with forcing unpopular technology on other users.

In general, I believe that most Visual Studio developers don’t take advantage of extensions as much as they should. Many developers could be more productive with an IDE that was customized to their preferences or needs. If you have never installed an extension, go out to the gallery and browse for some that might help. If you google for lists of the best extensions, you will find some that will help you. The same goes for users of Visual Studio Code or Xamarin Studio, both of which have rich extension galleries of their own.

In the coming days, I’ll post about some of the extensions that I find to be incredibly useful that are not widely known. I’ll not post again about VsVim even though it is my favorite extension and it is very well written. But the fact is, if you are the kind of developer that wants VsVim, you probably already know about it. So I’ll try to highlight some extension that you may not know about.

Book Review of Fire In The Valley

2016-09-10T14:30:00+00:00

As software developers, we must be dedicated to continual learning. It can be hard to find time to keep up with things we need to learn. Something that I have found to be helpful is to listen to audio books. Guys like Cory House and John Somnez have done a good job extolling the virtues of listening to audio books on your commute. I’m a big fan of Audible.com and have learned much from books in their catalog, but not many of them are very technical . So I was very excited when I found out the fine people at The Pragmatic Bookshelf were starting to publish audio books. I recently finished listening to Fire In the Valley. While it is not technically a programming book, it did help me understand more of the history of our industry.

The Package

First of all, let me say the purchased product was great. The recording quality was on par with the quality of books on Audible. This is important because if you are going to spend hours listening to something, you don’t want to be annoyed by poor quality audio. Also, the content comes in DRM free MP3s! So you are not locked into a format or a device. The Pragmatic Bookshelf publishes their content DRM free, and that always makes me more willing to purchase it.

Content

The book itself was great. It more or less covers the history of the personal computing era, which the book defines as the time from the launch of the Altair to the launch of the iPad. The story follows the prominent companies and the individuals of the time. The chapters group them into categories like hardware vendors, software vendors, publications, and retailers.

The Good

I really enjoyed reading about the creation of the Altair and the impact it had on the computer industry. I also really enjoyed reading about the Homebrew Computer Club and its importance. But being a software developer, my favorite sections were the ones about the prominent programmers from the early PC days.

It was interesting to read about how Bill Gates and Paul Allen got started in the industry. It was also interesting to see how Larry Ellison become the power broker that he become. As a C# developer, I really enjoyed reading about Anders Heilsberg and the early work he did with Pascal and Delhi.

The book does a good job of retelling the story of Apple. It is probably the most interesting story of the PC era. As a former Palm OS developer, I enjoyed reading the section on Palm and how it fit into the PC industry at the time.

Overall, I appreciated how the book explained how many of the prominent computer and software companies to today grew from startups into the powerful corporations they are. I definitely felt like the book gave me a better understanding of our industry.

The Bad

There was very little I didn’t like about the book. But I did feel like the section on IMSAI dragged on for a long time, especially considering the relatively small long-term impact the company had on the industry.

The only other thing I didn’t like was that the book didn’t go into the technical details of the machines and software it described. It’s not so much a flaw in the book, but a choice the authors made. As a developer, I would have liked to know more about the details.

Other Recommendations

If you are interested in this book there are some other very good audio books that are available on Audible.

The Innovators by Walter Isaacson covers the development of the computer from a broader level. It starts much earlier, including an interesting in-depth look at Charles Babbage and Ada Lovelace. It covers much more of the early history of ARPANET and the internet.

Steve Jobs by Walter Isaacson covers much of the same material as Fire In the Valley, but it obviously covers it from Jobs’ perspective. It explores more of the motivations of this fascinating man that was so influential in our industry. Reading (or really listening) to this book made me want to learn more about Steve Wozniak, which is what led me to read iWoz by Steve Wozniak. Wozniak was more of a pure engineer than Jobs. As a coder, it was fascinating to me to hear the story from his perspective and I enjoyed it more than the biography of Jobs.

Conclusion

If you have a commute, or if you have some time in your week where you are doing something relatively mindless, such as mowing the lawn, you can use that time to be learning. The fact that this book is available in audio form means that you could get through the whole thing at times that might otherwise be wasted. It would be a valuable read of its own merit. But the fact that you can listen to it while you drive makes it even more valuable.

Driving Canopy Tests with TypeProviders

2016-08-29T14:30:00+00:00

One of the most amazing features in F# is Type Providers. They allow you to access data sources in incredibly easy ways. You can query data from databases, read data from CSV files, or consume JSON data from an API. I’m not going to go into depth here on Type Providers, but if you are unfamiliar with them, you should go check them out.

Often times when I am writing Canopy tests, I want to validate that some data from the database is being displayed on the screen. So what I need to do is get the data from the database and use it in a Canopy test.

To use a TypeProvider for data retrieval, the first thing you will need to do is include the FSharp.Data.TypeProviders assembly from NuGet. After that, this is all of the code that you need to start getting data.

That is it. There is nothing else to do. You have strongly typed classes based on the data in your data source.

In this example, I am using the data from the Crisis Checkin database. Now that I have queried the list of disasters, I could easily right a test that ensures that all of the disasters are listed on the site. Or I could right tests that ensure that volunteers can register for disasters that are currently open.

Driving tests from the database allows the tests to use relevant information that stays up to date with the web application under test. There is no need to maintain a separate data source for the test data, it can easily directly from the database.

Validating List Sorting with Canopy

2016-08-19T14:30:00+00:00

One of the things that F# is particularly good at is dealing with things as lists. So when I had a Canopy test that needed to verify the elements on a page were sorted correctly, I looked for a way to leverage F#’s strengths. I am primarily a C# developer, so my first instinct was to get the elements, loop through them and check the ordering. But after some digging I came up with this solution.

There is a lot going on in a short chunk of good. Line 6 defines the canopy test. Line 7 calls a function I have defined elsewhere to log in the administrator user. Line 8 uses canopy’s ability to use CSS selectors to retrieve the elements in the rightmost column of a table. This returns a list of IWebElements. Line 9 uses the F# pipe forward operator to pass the result of line 8 into a method that gets the text of each element. The resulting list is passed to the isSorted function.

The isSorted method uses some of F#’s sophisticated list operations. The Seq namespace contains F#’s methods that operate on sequences. The pairwise function takes a list and returns a list of all of the pairs in the list. For example:

a,b,c,d,e becomes (a,b),(b,c),(c,d),(d,e)

The forall method takes a function and returns true if the the function is true for all of the items in the list. In this case, the list is the list of pairs returned from pairwise.

The result of isSorted is passed to Canopy’s is method on line 11. The is method determines if the test passes or fails.

Thanks to @ReidNEvans who helped me clean up this code. He also pointed out that I could have used the Fold function to validate the sort. Which I may explore in a future blog post.

One of the things I love about Canopy is that you don’t need to know much about F# to be productive with Canopy. But if you do know F#, you have it’s full power at your finger tips.

Prefix and Postfix Increment Deep Dive

2015-12-08T14:30:00+00:00

Recently a read a blog post by Eric Lippert about the top ten things he wished had been designed differently for C#. If you have not read it, you can find it here. It is worth reading in its entirety. The section that got me thinking was “#3: I rate plus-plus a minus-minus”. In it, he makes some very solid points about his dislike of the increment and decrement operators. Firstly he points out that the increment operator can easily be replaced with “x += 1;”. More importantly, he shows that the increment operator has two purposes, to return a value and alter the value of the variable. So by it’s very definition, the expression has a side effect.

But the following statement is the one that grabbed my attention.

Next, almost no one can give you a precise and accurate description of the difference between prefix and postfix forms of the operators. The most common incorrect description I hear is this: “The prefix form does the increment, assigns to storage, and then produces the value; the postfix form produces the value and then does the increment and assignment later.”

This explanation, that Lippert is saying is wrong, is the exact way that I was taught that it worked. Granted, I was taught originally in C++, and this may be true for how C++ works. I have always assumed that C# worked the same way. Lippert is saying that my assumption is wrong.

He goes on to describe what it really does.

Why is this description wrong? Because it implies an order of events in time that is not at all what C# actually does. When the operand is a variable, this is the actual behavior:

Both operators determine the value of the variable.

Both operators determine what value will be assigned back to storage.

Both operators assign the new value to storage.

The postfix operator produces the original value, and the prefix operator produces the assigned value.

So what he is saying is that both the increment and decrement operators do the arithmetic first and later return the value. But I was not satisfied with the text description, I wanted to know how it really worked. I wanted to see how it worked in IL. So I wrote the following code.

The first method does some simple addition. It serves as a benchmark for understanding what is happening in the IL. The next two methods exercise the prefix and postfix unary increment operators respectively and assign the resulting value a different variable.

##Understanding IL##

In order to understand what is happening when the code executes, you need to understand a little about how IL works. IL, or intermediate language, is the language that C# is compiled to and the language that the CLR executes. So it is analogous to an assembly language for a virtual machine.

IL is a stack based language. In order to do any operations or assignments, values must be moved from registers to the stack. Addition is done on the stack. Assignment is done by pushing a value onto the stack and then popping it off the stack into a different variable. I admit I am not an IL expert. Much of what I know about IL comes from this Wikipedia page.

So I took the code I wrote, listed above, and compiled it in Visual Studio. Then I used Telerik’s Just Decompile to view the IL it generated. It is important to note here that I compiled the code in the Debug configuration so that no compiler optimizations would be used. If I had compiled in the Release configuration, these methods would have been optimized down to a single returned value.

##Examining the Code##

Let’s start by looking at the method that uses the long hand way to increment a number.

Lines 3 - 6 allocate space for the variables. Lines 10 and 11 perform the assignment into the “a” variable by pushing 10 onto the stack and then popping it off the stack into “a”. Lines 12 - 14 perform the addition by pushing “a” and 1 onto the stack and then adding them. Line 15 pops the sum off the stack into “b”. Lines 16 - 21 return the value from “b”.

The code illustrates how the stack is used in IL to assign values and perform arithmetic. The code is not equivalent to using a unary increment operator because the value of “a” is not altered. So let’s look at the code that uses the prefix form of the increment operator.

Lines 11 - 16 perform the addition. But the result is not stored in “b”, the result is stored in a temporary variable named “V_2”. Lines 17 - 20 assign the value from “V_2” into “a” and “b”. This illustrates where my misunderstanding came from. I assumed the assignment into “b” was done directly from “a”, but it is not.

Here is the code for the postfix form of the operator.

Lines 11 - 14 store the value ten into “a” and “V_2”. Lines 15 - 18 perform the addition and store the result into “a”. Lines 19 - 20 assign the value from “V_2” into “b”.

##Conclusion##

I think I understand what Lippert said about the common explanation being wrong. Because I assumed that “b” would be assigned from “a” for both forms of the operator, I made assumptions about when the calculation was done. Seeing how the intermediate variable is used makes it clear that the arithmetic is always done first. The real difference comes from whether or not the intermediate variable has the value from before the addition or after.

Postscript

I am sure some people will point out that these details are unimportant and they will generally be right. The IL code does what I thought it does, just now how I thought it did it. And as I said, if compiler optimizations are on, none of this is likely to matter. I wrote this to satisfy my curiosity, not to improve the performance of my code.

Go To Implementation

2015-12-01T14:30:00+00:00

Yesterday Microsoft released Update 1 for Visual Studio 2015. There are a host of great features that were added. There are other blog posts that cover all of the features that are included. I just want to highlight the one that I am most excited about.

In the previous versions of Visual Studio, you had the ability to right click on a method invocation statement and click ‘Go To Definition’. This would take you to the code that defines that method. This is a great feature, but with our modern development practices, it left something to be desired.

More and more, we are developing code that makes use of interfaces. This is for a lot of good reasons. They make the code more testable and they help developers encapsulate functionality. But if you were in Visual Studio and you clicked ‘Go To Definition’ on an object that was defined as an instance of an interface, you would be taken to the definition in the interface, not to the code that defines the method.

There is where the beauty of the ‘Go To Implementation’ feature comes in. Now if you click on the same method invocation, you get an additional option to go to the implementations of the method.

If there is only one implementation, you will be taken directly to it. If there are multiple implementations, you will be given a list that you can choose from.

This isn’t a huge feature, but it is one that I will use almost daily. While Resharper has had this feature for awhile, I am really glad that this is now directly in Visual Studio.

VelociRead

2015-10-19T14:30:00+00:00

So I had this idea. I had been using speed reading sites like Spreeder and Spritz. They accelerate your speeding by only showing you one word at a time. The words are always in the same place, so you don’t have to move your eyes. I found them to be a very helpful way to get through my daily reading list. The only problem was that at times I wanted to vary the speed at which I was reading. Both of those tools move from one word to the next at a fixed rate.

I often try to read while on my stationary bike. I thought it would be interesting to try to control how fast speed reader moved using the bike. This would allow me to get through less interesting parts of the reading by pedaling faster. Conversely, I could pedal slower if I wanted to read part of it more carefully.

I realized that the device on my stationary bike that showed my “speed” was connected to the bike by a wire with two pins.

In order to figure out what they do, I hooked by bike up to an oscilloscope. I don’t have a real oscilloscope, so I decided to try out some software I had read about on the Make blog called ‘Soundcard Oscilloscope’. It isn’t as nice as a real scope, but it did everything that I needed it to do.

Using the software, I was able to tell that it omitted a high pulse once per revolution of the pedals. So I figured if I could convert that pulse into something meaningful to my PC, I would be able to make the project work.

Since I only had one input, I decided to use a Digispark microcontroller. The Digispark also has a library that allows it to function as a USB keyboard. I programmed the Digispark to act as if a ‘j’ had been typed on the keyboard once per pedal revolution. I found a connector that matched the connector for the stationary bike. I soldered the connector to the Digispark, leaving a few feet of wire between them.

I created my own clone of Spreeder using WPF. Instead of advancing from one word to the next at a fixed interval, it advances at a rate that is proportional to how fast I am pedaling. I added keyboard shortcuts to move to the next or previous chapter. I also added shortcuts to move forward or back 50 words. If you care to see it, the source code is here.

Because of the additional keyboard shortcuts, I wanted to be able to have a keyboard near the bike. I removed the PCB from a cheap USB hub. I plugged the DigiSpark and another USB keyboard into the hub. I put the combination into an Altoids tin.

The results have been satisfactory. I have read several books this way. I am continuing to make some tweaks to the software to improve the reading experience. But overall the project was a success.

C# Developer's Impression of Swift

2015-08-09T14:30:00+00:00

As part of our continuous learning efforts at Aptera, we had a competition to see who could come up with the best implementation of Fizz Buzz in Swift. This was my first time coding in Apple’s new language.

The Simple Solution

At the beginning of my career, I was a C / C++ developer. So for my first implementation, I fell back on those roots.

See it in action

The first thing worth noting is that Swift is a low ceremony language. I didn’t have to implement a Main method. Whatever code is outside of other blocks gets executed. In this, case, just line 24. The next noteworthy thing for devs that primarily work in C based languages is that the semi-colons are optional. I included it on line 9 and omitted it on line 12. Both execute properly. While Swift is an object oriented language, it has functional elements. The first hint of that is on line 1, which declares a member with the “let” keyword. This makes it immutable, or a constant. Mutable variables are declared with the “var” keyword, demonstrated on line 9. In both of these declarations, you can omit the type specification. This is because the compiler can imply the type for the initial assignment.

This solution functions properly, but the implementation didn’t make much use of the more interesting parts of the Swift language. It is essentially a C solution that I wrote in a .swift file.

Using a Swift Switch Statement

The first language feature I wanted to explore was the switch statement. Swift makes use of pattern matching in it’s switch statements

See it in action

In this solution, the switch statements operates on the variable i. It matches using the wild card pattern “_” and a where clause. In this example, it is uninteresting because all of the cases match on the wild card. But you can see how it would be useful to match on a value and a condition. This could flatten out nested logic into a single selection statement.

Of lesser importance is the fact that the case blocks don’t require a break statement. The language assumes that you only want to execute on block. If you want to execute another block, you have to insert the “fallthrough” keyword. Another nice feature is the built in support for ranges. On line 4, I declare a range from 1 to max and iterate over it.

Another nice facet of the switch statement is that a single case can match multiple values.

See it in action

In this case, lines 9 and 11 match multiple values.

To really make use of pattern matching, I needed to move to a more interesting solution.

Tuples and Pattern Matching

See it in action

This solution makes use of on my favorite Swift language features, Tuples. C# has tuples as a generic class, but Swift has them as a language construct. This allows for some really elegant usages.

The first function returns a tuple of bool, bool indicating whether or not the value is a multiple of three and/or fire. Now the switch statement on line 7 can pattern match the tuple.

Another thing to note from this solution is that types are always declared with a colon and a type name following the identifier. On line 1, I declared the parameter type as an int. The return type of the function is the tuple of bool, bool. Declaring the return type with an arrow (“->”) was new to ne, but I found it very intuitive.

Closures

See it in action

As I mentioned before, Swift has functional elements. One of the hallmarks of functional languages is that they have functions as first class elements. So you can assign functions to variables and pass them as arguments to other functions.

One of the variations on FizzBuzz is to return Fizz or Buzz for numbers that contain 3 or 5 instead of being multiples of 3 or 5. Another variation is to parameterize which numbers to check for instead of hard coding 3 and 5 and paramaterizing the words instead of hard coding Fizz and Buzz.

In this solution I support both of those cases by passing in the text to return and passing in anonymous functions to check the numeric values. Swift calls these types of functions closures. I updated ListFizzBuzzResults to accept two closures which find the target values. ListFizzBuzzResults passes those same closures to the method named IsFizzBuzz. IsFizzBuzz performs the checks for the target values.

This allows me to control how the program executes by passing different closures to ListFizzBuzzResults. On line 25 I declare and pass closures that check for multiples of three and multiples of five respectively. On line 28 I declare and pass closures that check for values containing 3 or 5. I can have either functionality simply by changing one line. I could easily have some combination of the two cases as well.

Declaring the closures is very terse. You wrap the code you want in the closure in braces. You identify the arguments with a dollar sign and their position number. For example, you use “$0” to identify the first argument.

To make this solution follow the functional paradigm, I also changed line 8 from a for statement to a map. I put the switch statement in a closure which gets passed as an argument to map. In this case I use the other closure syntax that allows me to define the parameter names instead of just using $0.

Overall Impressions

Having only spent a few days with it, I really enjoyed the Swift language. The syntax is terse, but sensible. It has many of the features of the most modern languages without felling entirely foreign. It allowed me to mix some functional features into object oriented code.

I suppose the highest praise that I can give it is that I hope to write more of it in the future.

Visual Studio 2015 Best Kept Secret

2015-07-23T14:30:00+00:00

With Visual Studio 2015 now fully released, I wanted to highlight one of my favorite features that isn’t getting enough press.

My primary development machine is a laptop. Most of my development is done in the office, but I frequently do my development at a client site. When I am in the office, I want to take advantage of all three monitors, putting Visual Studio tool windows across all of my monitors. But I need to be able to quickly switch back to having my entire IDE back on my laptop screen only.

In the past, there were plugins like Layouts O Rama that allowed you to save and restore window layout configurations. Now this functionality is baked right into Visual Studio. In the Window menu, there are four new options to manage window layouts.

Working without external monitors, I arranged all my tool windows where I wanted them. Then I clicked the Save Window Layout button. in the dialog that popped up, I named the layout “laptop only”. Later I connected my external monitors. Again, I manually arranged my tool windows where I wanted them and then saved the layout.

The next time I switched back to working on my laptop by itself, I clicked Apply Window Layout and selected “laptop only”. This put all of my windows back to where they were when I saved the layout.

It can also be useful to save layouts for specific tasks. For example, I have one layout saved specifically for editing RDLC files.

This isn’t a big ground breaking feature, but it is something that I use all the time and something that makes my life easier.

Using Canopy To Test Responsive Design

2015-06-25T14:30:00+00:00

I love Canopy for writing integration tests for web applications. One of Canopy’s best kept secrets is resize. It allows you to resizing the browser to represent different screen sizes. For example, the layout for my blog has some responsive elements. If the browser is as narrow as it is on a phone, the search box at the top will go away and the home link will be moved from the top bar into a menu. That menu has a button that will only be displayed on smaller screens.

Lets say I want to write two Canopy tests to make sure the button is hidden when the page is viewed on a laptop and it is displayed when the page is viewed on a phone. You could use the following code.

"The navbar button is hidden for a laptop screen" &&& fun _ ->
	resize (1024,768)
	notDisplayed "div.container a.btn.btn-navbar"

"The navbar button is displayed for a phone screen" &&& fun _ ->
	resize screenSizes.iPhone5
	displayed "div.container a.btn.btn-navbar"

In the first test, I call the resize method passing a tuple representing the minimum laptop screen that I want to test. This will resize the browser before executing the assertion on the next line.

In the second test, I make use of one of the constants that are defined by Canopy to automatically resize the browser to a know phone screen size.

Running these two test will give me confidence that the page is responding to various screen sizes. It is that easy and it can be very useful.

Commodore Vic 20

2015-04-12T14:30:00+00:00

I stopped by the Salvation Army last weekend to make a donation. When I went into get a receipt, I noticed that they had a Commodore Vic-20 on the shelf. I’ve heard about this machine many times on the Hello World podcast and was instantly fascinated by it. When I realized it still had the original manuals with it, I bought it.

Some of the things that are advertised on the packaging are quire humorous. It has a “typewriter-style keyboard.” When was the last time that was listed as a feature of computer. It was “the friendly computer.” Which is true if you consider booting straight into a BASIC interpreter to be friendly. I would still consider this a feature (albeit with a different language), but I assume I am in the minority here. It came with “arcade game excitement.” Technically this was true. More on that later.

It has the classic Commodore construction where the entire computer is housed under the keyboard. It was also manufactured before keyboard layouts were standardized. All of the letters are where you would expect them to be, but after that there are major differences. As a C# and JavaScript developer, I noticed right away that the semicolon isn’t were modern keyboards have it. Maybe most notably, it has a key for pound (£), which modern keyboards don’t have. In fact, I had to insert a ‘special character’ to include the symbol in the previous sentence. Noticeably absent are the arrow keys and the escape key. The biggest disappointment was that the keyboard use rubber membrane switches and not something with a spring in it like the Apple II had or the IBM model M keyboards had.

One of my favorite things about this purchase is that it came with the original literature. Look how much fun that family is having working on the spreadsheet for their family budget.

It only takes seven pages before the owners manual is telling you how to write your first program. The rest of the 164 page owners manual is a introduction to programming.

Now when box said it came with “arcade style fun”, what it meant was that it came with a book of source code for a bunch of different games. So what you had to do was boot up the computer, type in a few hundred lines of code perfectly, then you could play the game. To be fair, there were cartridges you could buy or a cassette tape drive you could use to load games, but those were extra. My purchase didn’t come with either.

I was able to hook it up to my TV and it turned on right away. But I immediately noticed that about half of the characters did not display correctly.

As I typed, some of the characters displayed properly and some displayed as solid blue boxes. Interestingly, when you line the characters up in ASCII order, 4 in a row will display properly and the next four will fail. This pattern holds for all characters. In fact an character whose ASCII has the 3rd bit (the one whose decimal value is 4) set to 1 would not display properly. If I typed in the code, it would execute perfectly, other than the display. So the problem is somewhere in the display circuitry.

I opened up the device to see if there was anything obviously wrong. I didn’t see anything that looked to be causing the malfunction, but it was interesting to see how it was built. This was before everything was built with surface mount electronics.

All of the components are full sized, through-hole soldered components. The chips aren’t all combined into integrated circuits. In fact many standard TTL chips are on the board. There is a very standard looking 555 chip near one of the expansion ports.

Because it doesn’t work properly and because any software I could run on this I could easily run in an emulator, I’m going to dissemble this one and turn it into something else. More on that in a future post.

ScriptCS-Arduino controlled Nerf gun

2015-01-19T14:30:00+00:00

One night I was hanging out with my sons. We were talking about robotics and one of them asked if I could build a robot that would fire a Nerf gun. So we started out with a bunch of parts I had on hand to see if we could do it. I mounted the Nerf gun to a piece of peg board and then set out to see what I had that could pull the trigger. My first attempt was to use a motor that was geared way down to pull back a string that was attached to the trigger. I used an Arduino with a motor sheild to control the motor. That worked but it was slow. Since the boys were bored with a Nerf gun that took 5 seconds to fire, we called it a night.

To make it faster, I ordered a 24V Solenoid from Adafruit. I had a AC adapter that I salvaged out of an old printer that I used to power the solenoid. To control the higher voltage, I used a TIP120 transistor I got from RadioShack. Now I could fire the Nerf gun almost instantly with the press of a button.

To make it more interactive, I started controlling the Arduino with the ScriptCS-Arduino library. This made it controllable from my laptop.

A few weeks later, I came across an old 5 disk CD changer. I decided to use this to rotate the gun horizontally, allowing me to aim it in different directions. I took out most of the electronics and soldered longer wires onto the motor that spins the tray. I wired the motor up to the 24V source I was using for the solenoid and controlled that with another TIP120.

In the original version, the wires went straight from the Arduino up to the solenoid. This was functional, but it meant that if I spun it around too much, the wires would get twisted or unplugged. This got old after a few weeks.

I decided that I wanted the gun to spin freely, but I needed away to get the control voltage up to the solenoid. Inspired by Ben Heck’s Steampunk Persistence of Vision Display, I mounted two metal rings on the bottom of the CD tray.

On the base, I mounted to spring mounted contacts that would create a connection to the rings above. The allows the circuit to be complete no matter what position the CD tray is in. If I had a 3D printer like Ben Heck, the build would have been much cleaner.

Over time, I continued to make improvements. I got a prototyping shield for the circuit. I got some standoffs and mounted the whole thing under the CD tray.

One problem that I encountered was that the motor drove the CD tray with a rubber band. If I put the full 24V on the motor immediately, the band would often slip. So what I decided to do was use the PWM functionality of ScriptCS-Arduino to spin the motor slowly at first and then accelerate. The made the motor spin much more reliably.

The full source code is available on GitHub in a file named NerfGunExample.csx

It is far from perfect. I am putting a lot more weight on the CD than it was designed for, so it will certainly break at some point. The control for the motor is very crude. At the very least, I should use an H-bridge so that I could spin the gun in both directions. It would be even better if I used a stepper motor so that I had much more precise control over the movements.

But it is fun to play with. It demonstrates many of the things about ScriptCS-Arduino that I really like. It is fun to play around with. Here it is in action.

ScriptCs-Arduino Controlled Nerf Gun from Eric Potter on Vimeo.

C# and Microcontrollers

2015-01-04T14:30:00+00:00

There are plethora of microcontrollers available today. If you want to write your microcontroller code in C#, you have a couple of options.

First lets look at how a standard Arduino works today. Standard Arduinos like the Arduino UNO are programmed from a computer. Once the code is loaded, they are fully autonomous from the computer. The Arduino has it’s own processor and storage, so it doesn’t need the computer to run.

There is an open source IDE for Arduino that runs on your laptop. The code that write is in C. (Technically it is Processing, but it is very close to C.)

There is another board that you can purchase called the Netduino. It is more powerful and more expensive than an Arduino.

The code you write for the Netduino is C# and you use Visual Studio as the IDE. The Netduino has a processor that is powerful enough to run the .Net Micro Framework. So the code you write executes write on the Netduino.

If you want to write C# code that works with an Arduino, you can use the ScriptCS Arduino library. In this case, you write CS code that executes on the computer that controls the Arduino. The Arduino must have a USB connection back to the computer as as the code executes.

You load a program on the Arduino called Firmata. It comes as one of the examples with the Arduino IDE. Once this is done, you can write C# code that executes in ScriptCS. Your C# code sends commands to the Arduino via the USB connection.

What is so exciting about Roslyn?

2014-04-30T14:30:00+00:00

I had someone ask me recently why I am so excited about Roslyn. It is a fair question because while Roslyn is an amazing technology, all of the benefits are not immediately obvious.

What is it

If you are not familiar, Roslyn is the code name for Microsoft’s next generation .NET compilers. But more than that it is a library that exposes the compiler’s internal data structures and a framework that enables customization of the compilation process. This is a radical departure from the current C# and VB compilers, and the vast majority of all compilers. Most compilers are black boxes. Code goes in one side and an executable comes out the other side. At a conceptual level, a user might know what is happening under the hood, but there is no access to the inner workings.

Roslyn changes that. Roslyn give you the ability to take a source file as input, but get access to the syntax tree once it has been parsed. It gives you the ability to view the semantic information about a syntax node once it has been generated. Having analyzed the source, you have the ability to alter the IL before it is generated.

So What

But the question remains. So what. Why is this so exciting? With direct access to the syntax tree and its semantic information, we can pro grammatically understand our code easier than ever before. As programmers, our code is our poetry. It is the blueprint for our designs. It is how we communicate with the machine. It is one way that communicate with other programmers.

One of the great things about unit tests is that they allow us to test our code with code. In a similar way, Roslyn allows us to understand and analyze our code with code. That is very powerful.

None of the functionality come out of the box, you have to write it. Conveniently, you are very good at writing software. So you have to find an itch and then scratch it with Roslyn. You can create tools to make the development process easier. If you share those tools, it benefits our entire community. Creating tools is fundamentally what we do. With Roslyn, we can create tools for ourselves.

Even if you don’t use Roslyn to create tools, you will benefit from it. Roslyn will enable Microsoft to more easily make improvements both to the C# and VB languages and to the Visual Studio experience. Commercial tools also have access to Roslyn, which will allow then to create more and better tools. And it will allow them to do it faster.

As an example of what you code do, Joe Hummel did a great presentation at the Chicago Code Camp where he created a Visual Studio extension that would generate a compiler warning if there was code that had an empty catch block. It could have been easily altered to generate a compiler error. It also had the ability to suggest a code fix. So if you had an empty catch block, you would get a tool tip that add a throw to the catch block for you.

In the Object Oriented Design class I taught this semester, I had my students use Roslyn to look at C# source files and gather the number of statements and max nesting depth of each method. Using this information they could suggest which methods in the file were too complex and in need of refactoring.

That example highlights one of those inception moments Roslyn enables. The code can analyze itself. The rabbit hole gets deeper when you realize that Roslyn can compile itself.

So let’s get back to the original question, why I am so excited about Roslyn? I am excited about the tools you and I are going to build with it.

Getting started with ScriptCS-Arduino

2014-04-04T14:30:00+00:00

Arduinos and PCs

I love making things with Arduino boards. The documentation is great. There are plenty wonderful accessories. And best of all, the community is amazing. For projects that are designed to be autonomous, it is hard to beat the Arduino.

But many of my projects are designed be used while connected to my computer. The Arduino supports serial communication which enables the Arduino and the PC to communicate. Some of the newer boards are powerful enough to emulate USB keyboards. The Arduino can simulate key presses, and the PC receives them as if a USB keyboard was sending the commands. This is how DigBug and VelociRead work. This is easy to develop, the but communication only flows from the Arduino to the PC. The Arduino can also open a serial connection which enables bidirectional communication. The drawback to this is that it takes more work to develop.

ScriptCS-Arduino

Enter ScriptCS-Arduino. This fantastic library allows you to rapidly develop code in C# that controls the Arduino. It was developed by Luis Rudge and is fully open source. It makes use of ScriptCS to execute code on the PC and it uses the Firmata library to control the Arduino.

The execution model with ScriptCS-Arduino is different from standard Arduino programming. Normally, you would write all of your Arduino code in the Arduino IDE and write it to the Arduino board. With ScriptCS-Arduino, you use the Arduino IDE to load the Firmata code onto the Arduino and you do the rest with ScriptCS. Essentially, the Arduino board becomes a slave board. All of the important code runs on the PC.

Installing the Software

In this post, I’m going to show you how to get started with ScriptCS-Arduino, specifically on Windows PC. The first thing you need to do in order to use ScriptCS-Arduino is to install ScriptCS. The full details are available at http://scriptcs.net/. But if you have Chocolatey installed, you only need to use the following command. (If you don’t have Chocolatey installed, you should check it out. It is very useful.)

cinst scriptcs

Next you will need to have the Arduino IDE installed. You can install it with this Chocolatey command.

cinst arduinoide -Version 1.0.5.20130613

Or you can get it from the Arduino software page. With your Arduino connected to the PC, you need to load the Firmata library on to the Arduino. You don’t need to download it because Firmata comes with the Arduino IDE. In the Arduino IDE, select “Standard Firmata” from the examples menu.

Load this software onto your Arduino. Make sure you have the right board and Serial Port selected in the Tools menu. Then click the Upload button.

Next you need to setup ScriptCS. One of the most powerful things about ScriptCS is that it is fully integrated with NuGet. The means that you can easily install ScriptCS-Arduino NuGet Package. ScriptCS-Arduino is a NuGet package specially designed for ScriptCS called a ScriptPack In your command shell of choice navigate to a directory where you want to work. Type the following command.

scriptcs -install scriptcs.arduino

This will install the NuGet package in your current directory. ScriptCS executes .csx files. Your .csx files in this directory can use the NuGet package you just downloaded. Here is a simple example of .csx file using ScriptCS-Arduino. It simply blinks and LED, which is basically the hello world of Arduino programming.

Writing the Code

This script assumes that there is an LED with it’s anode (positive side) connected to pin 9 on the Arduino and it’s cathode (negative side) connected to ground.

Line 1 of the code imports the ScriptCS-Arduino ScriptPack. Line 2 creates an object that represents the Arduino board. Line 4 creates an instance of the Led class. The Led class is part of the ScriptCS-Arduino ScriptPack. It simplifies the code necessary to control an Led. Lines 6 through 9 dictate that the LED blink for 2 seconds and then stop. Line 11 cleans up all of the resources.

Possibilities

This example is trivial and it could have been done entirely in the Arduino IDE. But ScriptCS- Arduino opens up a host of great possibilities. The biggest opportunity is that you could integrate your Arduino into projects on your PC. For example, instead of just randomly blinking and LED for 2 seconds, you could run the script as a part of your build process and blink a red LED if there is a build failure and green LED if it succeeded. Because of the power of Arduino, you are not limited to LEDs. You could sound and alarm, spin something, or even fire a Nerf gun. The possibilities are nearly endless.

Another fantastic thing about ScriptCS-Arduino is that it allows you to prototype things very quickly. You can write little a script to try out just about any behavior of the Arduino. For those of us who know C#, it makes it faster to be working in a familiar language. You can use the ScriptCS REPL to make your Arduino exploration even more flexible. I’ll explore this topic more in a future blog post.

Get Started

You now know enough to get started. Use your imagination. Building something cool. Have some fun!

VelociRead

2014-03-02T14:30:00+00:00

So I had this idea. I had been using speed reading sites like Spreeder and Spritz. I found it to be a very helpful way to get through my daily reading list.The only problem was that at times I wanted to vary the speed at which I was reading.

So I decided to write an application that would serve as the speed reading and then connect it to my stationary bike to control the speed. I wrote the app in WPF. I connected my stationary bike to a DigiSpark. The DigiSpark connects to the PC as a USB keyboard. It detects when the pedals complete a revolution and sends a keystroke to the computer.

USB Switch Cable

2013-09-08T14:30:00+00:00

The Problem

When working on Arduino projects, it is often necessary to plug and unplug the USB cable repeatedly. This is especially true with the Digisparks when need to be unplugged and then plugged in every time you program them. This certainly isn’t hard, but it is a nuisance.

The Solution

To make this process more pleasant, I wanted a way that I could disconnect a USB connection with a switch. I figured that if I used a double pole, quadruple throw switch, I could virtually unplug the USB cable.

The Build

The construction was really quite simple. I had the switch left over from some previous project. The female end actually came out of a printer that I disassembled. The male end of the cable was from an extra USB cable, which are easy to come by.

I cut the cables and exposed the wires. Using a multi-meter, I figured out which wire needed to be connected. I also determined which pins needed to be used on the switch and soldered the wires to the switch.

A Tic-Tac case was just about the perfect size. I used a utility knife to carefully cut out the holes that I needed. Then I riveted the switch and the female connector to the case.

Results

The cable works as expected and is very useful for Digispark projects. It is also useful for projects where the Arduino simulates a keyboard or a mouse. The build could have been prettier if I hadn’t only used parts I had laying around.

At the end of the day, it is a quick, cheap and easy project. Most importantly, it is useful.

Notes to a Software Team Leader

2013-09-06T14:30:00+00:00

In the last 2 years, I have made the transition from programmer to technical team leader. I found myself unprepared in some ways for my new responsibilities. I knew how to improve my technical skills, but I wasn’t sure how to get better at my newly required soft skills. Needless to say, I was very excited when I heard about Roy Osherove’s book Notes to a Software Team Leader.

The book is spot on when it says “Most of us weren’t taught how to do this type of work.” We want to be good at it. Our teams and our software are depending on us being able to lead. This book provides advice for ever team leader on how to become a better leader. It gives practical and useful ideas about how to manage your time and efforts as well as how to interact with others.

One of the key themes in the book is learning. As a team leader you need to learn to be better in your role. A key facet of that is facilitating ways for your team to grow. The book examines how you can improve in these areas.

The book concludes with a series of essays from thought leaders in the software industry. Each other tells what they would like to say to someone who is brand new in a software team leadership position.

I was very pleased with the book. The information was useful and applicable. I am better equipped to do my job for having read it.

Lessons From iWoz

2013-08-26T14:30:00+00:00

#iWoz# Like many people I read Walter Issacson’s biography of Steve Jobs soon after it was published. It was a great read and Jobs was certainly a fascinating individual. But for me, the most interesting person in the book was Steve Wozniak. Jobs was the visionary that drove Apple, but in the early days, it was Wozniak that brought the vision to fruition. Wanting to know more about Wozniak, I recently read his autobiography titled iWoz.

The full title of the book is iWoz: Computer Geek to Cult Icon: How I Invented the Personal Computer, Co-Founded Apple, and Had Fun Doing It. This explains a lot about why I loved the book. The idea that inventing and creating this is fun is largely why I got into engineering in the first place. Woz exemplifies this mindset that it is fun to make new things. And he made one of the most brilliantly engineered devices of the modern era.

Woz was primarily as hardware engineer. He did write quite a bit of software as well. Being a programmer, I read the book thinking about what lessons I can learn for him. This is what I took away.

Loving What You do

It is clear from the book that one of the reasons that Woz was such a great engineer was that he loved it so much. He truly enjoyed building electronic gadgets. In high school, his hobby was studying the designs of mini computers and trying to figure out how to rebuild them using fewer chips. He build his first computer, which he called the cream soda computer, just for fun. He dedicated a great many hours to becoming a better engineer, and was able to do so in part because he loved it.

Thinking About Your Users

Early in the book, Woz describes himself as “an engineer who worries about people a lot.” The Apple I and the Apple II were both brilliantly engineered computers. But the reason that they started the PC revolution was that a person could use the computer without being an electronics wizard. Woz thought about what the user would want and built something that met their needs. Steve Jobs also obsessed with this in everything he did. If fact that was one of the main lessons I took away from his biography as well.

As software engineers, we build tools. We are only successful when our users are successful. We must be focused on how are users will benefit from our software and that it will be intuitive for them.

Craftsmanship

Woz cared deeply about the quality of his work. You can see the fruits of this whenever you see an Apple II today. Even though the Apple IIs are roughly 30 years old today, the chances are that if you can find one, it will still work. In chapter 20 he says “And it is this reach for perfection, this striving to put everything together so perfectly in a way that no one has done before, that makes and engineer or anyone else a true artist.”

As software engineers, we ought to care about the quality of our work. Will our work stand the test of time? Will it be valuable to our users for years to come?

Design

Whenever Woz was building something new, he always spent a lot of time in preparation before he actually built it. He would study the chips he was thinking about using. He would design and redesign the circuits on paper. When it was time to build them, he already knew they would work.

In my experience, we often jump to coding too quickly. Often it is more productive to work through a problem on paper. Sometimes, the greatest tool that a programmer has is a large white board. Having a good plan leads to better software. This is also one of the reasons that Test Driven Design is beneficial. If you have written the test, you have done design work.

Preparedness

Woz was a brilliant engineer. But it is impossible to believe that his success was not a combination of his talent and fortunate timing. He was coming into his own at a time when microprocessors and ram became affordable to individuals. At the same time, floppy drives became available. Mainframes and minicomputers have created a base of computer programmers that were able to create software for the personal consumer market. The confluence of these events enabled the Apple I to be the success that it was.

But what set him apart at the time, was the he may have been the only person alive that was prepared to build the personal computer. The parts were available to everyone, but he was the only one with knowledge of all the parts. He had experience building simple computers from the time that he built a simple computer out of integrated circuits. He had experience with keyboard input and video output from building his own terminal. He also had experience with video output from working on a early device that would be considered a VCR. These were project he had done in his spare time just for fun. He also had experience with booting a computer from ROM from the work he did at HP on calculators. He had experience programming from his time in collage, some previous jobs, and the time he and Steve Jobs developed the game Breakout for Atari.

His experience came from his education, his professional work, and his personal projects. Likewise, as software engineers, we need to be constantly broadening our skill set. Some of that will be done through education and projects at work. But learning things on our own time is also very valuable. Side projects give us an opportunity to delve into technologies that we enjoy and technologies that are cutting edge, if unproven.

Doing projects just for fun or contributing to open source projects can be very valuable to our skills sets. It gives us an opportunity to work with tools and technologies that are outside of our comfort zones.

Often times, when an opportunity arises, there isn’t time to learn a new technology on the spot. We need to be ready in advance. Or sometimes we may miss an opportunity to utilize the best tools if we are not aware of them.

Learning

The road to preparedness is learning, a subject that Woz visits repeatedly in the book. One idea that I was impressed with was when he stressed the importance of leaning gradually, “one tiny little step at a time”. There is no shortcut to mastery. You must pay your dues at each step. Talking about some of the early projects he did, Woz says “I learned to … concentrate on the step I was on and to try to do it as perfectly as I could when I was doing it.” To really learn something you can’t just skip to the end, you have to “do the in between steps.” He also says “You can’t teach somebody two cognitive steps from where there are.” The corollary being that we can’t teach ourselves something two steps from where we are.

Woz learned both by reading and by doing. We should do the same. We should be reading about topics that are just beyond our current skill set. And we should be dabbling in projects that push us a bit further than we have ever gone before.

The iTunes Rating Device

2013-07-19T14:30:00+00:00

I built this box so that I could easily change the rating of songs in iTunes. The idea is that if iTunes is playing while the window is mimimized or hidden, I want to know the current rating and change it without switching to the iTunes window.

The box has 5 LEDs that display the number of stars currently assigned to the track. Then there is a nob that increases the star rating when it is turned to the right and decreases the start rating when turned to the left. Lastly, there is a red play/pause button.

There is software that runs on the computer that recieves these inputs. It forwards the commands on to iTunes via it’s COM interface.

The box is built around a phidgets board. (http://www.phidgets.com/). Phidgets are unique in that all of the processing is done on the computer and the board is simply an IO device. The board is conected to the computer by USB. The program is written in C# using the Phidgets libraries.

The Phidgets hardware was very easy to use. It does have the limitation of needing to be teathered to a PC. But for this project, that was perfect.

The Right Size Wrench

2013-07-19T14:30:00+00:00

My grandpa was a mechanic for Catapiler machinery. He was one of the finest and hardest working men I have ever known. Ever since I was little I was fascinated by his tool box, especially the wrenches. He had a wrench for everything. Little wrenches, big wrenchs, socket wrenches, cresenct wrenches, allen wrenchs, and open-end wrenches. My favorite was a 1 1/2 inch open-end wrench that weights more than two pounds.

Grandpa didn’t collect wrenchs because he liked something about the wrenchs, he collected them because they helped him solve real problems.

I am not a mechanic and have no need for a 2 inch open-end wrench. But my job is to solve real problems. I am a software engineer and my tools are things like programming languages, compilers, static analyzers, and white boards.

In the same way that my grandpa always used the right size wrench, it is my job to use the right tool for each task I encounter. I don’t use a tool because of how popular it is. I don’t use a tool because it is what worked on the last task. And I don’t use a tool because it is the tool that I know the best. I use a tool to get a job done.

It’s not enough to continue to use the same old tools. You have to learn, adding new tools to the tool box.

I don’t just use tools, I build tools. Fred Brooks once described a programmer as a toolsmith. He said “That swordsmith is successful whose clients die of old age.” The software I build is only successful if it is a useful tool for my users.

So that is why this blog is named Right Size Wrench. It reflects both the timeless craftsmanship of a job well done as well as the everchanging world of software development. Using the right tools to create the right tools.

Gear Clock

2013-07-19T14:30:00+00:00

Awhile ago, I got man hands on an old and broken printer, not the desktop kind, but large small office kind. I took it apart for parts. I was intreguid by the gear mechanism for one of the paper feeders. I decided to make a clock out of it.

There is a large gear that is marked out for the minutes and a smaller gear that is marked for the minutes. Because of the way the mechanism is built, the minute gear needs to spin counter clockwise. As the minute gear moves, it moves the gears and arms around it.

The clock is driven by an Arduino Uno with the Adafruit motor shield. Both the minute gear and the hour gear are driven by stepper motors that came out of the same printer. The steppers are much more powerful than they need to be, but I liked the idea of only using parts from the printer to build the clock.

DigBug

2013-07-19T14:30:00+00:00

As a programmer, I spend a fair amount of time debugging code. One day I realized that the four primary debugging commands (run, step over, step in, step out) are roughly analogous to the four directions of a joystick (up, down, left, right). It is also useful to have buttons to stop and toggle breakpoints.

I found an arcade joystick and two arcade style buttons. They are mounted in a wood encloure that is an homage to an arcade machine. The joystick and the buttons are connected to a Teensy microcontroller. (http://www.pjrc.com/teensy/) The Teensy acts like a USB keyboard and sends the correct keyboard shortcut for each of the inputs. Because the computer sees DigBug as a keyboard, there is nothing to install on the desktop.

The Teensy microcontroller was great to work with. It can be programmed with the Arduino IDE, which is what I did. (You can get the Teensyduino libraries from their website.) It is small, so it can easily be embedded in a project like this. It was nice libraries to make the Teensy act like a keyboard or a mouse.

My wife created the artwork on the side. Each side has an image that looks like one of the Visual Studio debug icons dug out in a Dig Dug level.

The Joy of Making Things

2013-06-28T14:30:00+00:00

There is something fun about making things. It might me something fun made out of LEGOs. It might be something temporary made out of recycled materials. It might be something useful like a rain barrel. No matter what it is, there is something enjoyable about creation and something very satisfying about it’s completion.