If you ever programmed in a functional language, a dynamic language like Python or Perl, or even BASIC, you’ll likely be familiar with an interactive programming interface - the Read Eval Print Loop. In this series of articles I’m going to talk about options for running a REPL on .NET, and a little about why you might want one.
Since LISP the REPL has been well-used in the functional programming world, and similarly dynamic languages (less politely “scripting” languages) tend to provide an interactive shell. On the other hand languages in the C family historically tended to have a much more static view of the world. Sadly this lack has been inherited by C# - so what alternatives are there in the .NET world, and why would you want one?
Interactive prototyping & development
I include this first because it may (or may not) be the most important use of an interactive environment - but I’m not going to evangelise this here. You can take this to different degrees, but the basic idea is that you will be executing code in the REPL frequently during the development of your program - more frequently than you would recompile and run an entire app. The code being run in the REPL will tend to be a mix of updated definitions (e.g. new version of a function) and interaction with existing entities (more like testing).
Sometimes you can learn a new API from the wonderful documentation, but all too often it’s inadequate, or you just want to roll your sleeves up and see what it can do. An editor with good code completion gets you some of the way, but there’s no substitute for actually calling methods and inspecting data, and an interactive environment streamlines this process no end.
Similar but orthogonal to API exploration. There are custom data visualisation tools, but when you have a bunch of data to fetch, process and explore, why not do it from the comfort of your favourite programming environment? Take the data processing tools in the .NET framework, a charting or mapping library and explore!
For the purpose of this overview, I’m going to set a task to accomplish via the various
options presented. This will consist of fetching some data via the WorldBank API and
extracting some information. The data is the indicator “EN.URB.LCTY”, the population
in a country’s largest city (for 2009), and we’ll end up with a list of countries in
order of population (descending). There are no doubt many ways to do this (perhaps
“better” or “more native”), but in each case we’ll use the standard .NET
to download the data in XML format, and then XML to LINQ to parse the data.
The various implementations of this will tend to take a very similar format. Firstly we’ll add references and import the appropriate namespaces, then we’ll perform the download and extract a list of nodes - as uses of the same .NET API, this will look very similar in every case, a pure question of syntax. Lastly we’ll perform some actual list processing, and here the languages used can show their differences and native concepts a bit more.
It’s worth noting that I’m presenting examples of scripts as a whole, which may be somewhat misleading as to the use of the REPL. In each case I executed these examples line by line in the REPL, modifying and correcting as I went along, what you see in code snippets here are the collected final results of this evolution.
DLR - the iron languages
Microsoft implemented support for dynamic languages on .NET, via the Dynamic Language Runtime (DLR). There were 2 officially supported languages developed within Microsoft, IronPython and IronRuby; various other support was planned but shelved. As of 2010 official support ceased and the projects were handed over to the open source community: it now seems that IronPython is going strong but IronRuby is looking rather dead (but usable!).
IronPython comes out of the box with “IronPython Console”, the IronPython REPL in a console window:
This comes with standard command history etc, and you could run it in a better (non default) console, but to be frank it’s a little painful. A better experience is in Visual Studio 2010 or 2012 with Python tools. This gives you standard IDE features, syntax highlighting, and even better IntelliSense for both standard python libraries and .NET libraries.
Then you have the IronPython interactive window where the code is executed, again supporting easy editing, “send to interactive”, syntax highlighting, IntelliSense (this time dynamically based on the actual objects rather than static analysis).
Firstly we use the IronPython specific
clr module, adding the required assembly references
and in some cases importing the identifiers. Then the code looks rather familiar, to the C#
eye the only difference is a missing
new when constructing the
We can then define some utilities to extract the bits we’re interested from the XML.
The result of the
DescendantsAndSelf method is an
IEnumerable, this is exposed
as a python iterable, which we can use in a list comprehension to construct a list of value/country pairs.
Then the result:
As of 2013 IronRuby isn’t looking in the best of states. It’s open sourced, with some ongoing development, and should be considered “fully working”; but there still is no support in IronRuby Tools for VS2012. I like Ruby as a language but you would seem to need a strong reason to rely on IronRuby. Saying that, if you know Ruby and want to play in .NET, why not?
Some of you may be surprised at this entry. However PowerShell is not only an interactive environment
(by virtue of being a shell!), but also a full .NET citizen. One can natively create .NET objects
New-Object cmdlet) and call their methods, though the syntax may be a little strange
at first. The other key point about PowerShell is that its command pipeline is not text-based
as per traditional shells, but consists of fully structured objects, meaning it really isn’t
(too) painful to pass these objects around and manipulate them.
Powershell runs in the console as standard:
However a richer environment is found in the PowerShell ISE.
The great thing is that now PowerShell comes with Windows as standard.
More assemblies are referenced by default, but here we sadly don’t have the benefit of abbreviating namespaces:
We can then pass the enumerable results of
DescendantsAndSelf through the PowerShell pipeline,
Where-Object, iterating with
Select-Object. I use the full names here, symbols
% and aliases may be easier to type
Here we chose to construct an object with named properties with
New-Object. We could
have just used a tuple, but this gave us the ability to sort nicely, and to output the
structured object in a readable fashion.
F# is unique on this list, for now, in being a first-class .NET citizen with full support from MS, and having a proper REPL in its official incarnation. F# is a powerful language in its own right (which you could class as hybrid functional/OO), and fully integrates with .NET. Perhaps it might take some getting used to for a C# or VB programmer, but I’d argue that it’s worth it (but then I am a fan).
F# enjoys Visual Studio support in the interactive environment. It’s slightly unfortunate that syntax highlighting and IntelliSense are not permitted in the interactive window, the appropriate methodology being to edit a F# script file or program and send snippets to the interactive window (alt-enter and alt-#).
Again we reference the required DLLs and open namespaces:
In the remaining implementation we see a list comprehension, similar to the python case, and also some pipelining, typical of F# code:
The big feature in the last edition of F# was type providers. These are basically just a fancy form of code generation, but the trick here is the Visual Studio integration. A type provider is defined for a particular resource and at edit time VS is able to provide IntelliSense for the generated types.
It so happens that one type provider that’s been included from the start and is now included in FSharp.Data is a World Bank provider. The DLL is referenced (either in a project or interactive session):
Then while in the editor, the type provider will enable intellisense based on actual data fetched from the WorldBank API:
You can see here that properties are generated automatically based on the data.
Unfortunately the WorldBank type provider doesn’t quite fit the bill of accomplishing the task set above - the ability to fetch all-countries data for a particular year isn’t exposed in this case. We could of course add this functionality for the benefit of programmer-kind, but in this instance we’ll make use of another type provider instead. The XmlProvider from FSharp.Data generates a strongly typed view of XML based on a sample document.
Firstly the type provider creates a type based on the name of the input file:
Then we can use this to parse the data (note XmlProvider here is a typedef of a type created by the type provider):
Then the data itself is typed with properties according to the XML:
The complete code is as follows:
As for C#, that shall be the subject of my next post, where I’ll cover various options, both “Open Source” and “Official”.
You can download the full source for the examples above from this repository.