How to get changed files from SVN between two revisions with PowerShell

As part of teaching myself PowerShell (and converting a legacy mess of perl scripts into something more manageable), I needed a way to export the files that were added or modified between two SVN revisions. After some searching, I came up with this PowerShell script: it takes a repository URL, a from and to revision numbers, and an output directory into which to export the files.

This script uses Resolve-FullPath cmdlet from the Carbon project. Turns out, PowerShell’s own Resolve-Path doesn’t work on files/paths that do not exist.

Here’s how it works:

  1. It executes svn diff command with –r switch, which takes a range of revisions, e.g. 1000:1050. The summarize argument shows only the high-level information, and xml outputs the data as XML.
  2. This XML is being queried with an XPath, extracting only the names of the items of kind ‘file’ which were ‘added’ or ‘modified’.
  3. For every such file, its relative path is being taken by subtracting the repository path from the full filename.
  4. The target (sub)directory for that file is being created, if it doesn’t exist.
  5. It then executes svn export command on the current filename, limiting it to the to revision, so that  the changes are taken only until that revision. The file is written in the relative target directory.

This is far from “idiomatic” PowerShell, but it gets the job done! Your improvements are welcome, feel free to comment on the Gist!

The 2-minute PowerShell intro for someone who “hates PowerShell”

Hi, I’m a developer, and I hate PowerShell*. For no reason in particular, PowerShell just never seemed that appealing for me as a developer, as it was always marketed towards sysadmins. And I never liked the syntax – all those dollar signs reminded me of PHP.

This recently changed, however. A client had a set of legacy perl scripts which nobody could maintain anymore – and I was tasked with rewriting them. Having recently spent some time writing a script for Boxstarter – an amazing tool that allows you to automate software and environment installations of new machines, powered by Chocolatey, in PowerShell, I thought that it could be a good fit for the task. I knew the language is powerful enough to allow the creation of those very useful tools, so I decided to give it a shot. I was not disappointed!

Luckily, you already have all the tools you need to get started! Every Windows machine comes with both PowerShell and PowerShell ISE (Integrated Scripting Environment) already installed.

PowerShell ISE
Figure 1: PowerShell ISE – an IDE for PowerShell with code completion (IntelliSense) and an integrated debugger.

Here’s what you need to know about PowerShell:

You can do most of your development and debugging in the ISE – no need to type individual instructions in the PowerShell console. It’s great as a REPL, but nothing beats an IDE. Tip: select a few lines in the ISE and press F8 – only those lines will be executed!

What’s so great about PowerShell?

It’s dynamic – every variable can be treated as anything, but also support explicit typing (which are checked at runtime). Variables are prefixed with the $ sign.

$my = "some string"
$my = 23

# Runtime error: Cannot convert value "23" to type "System.Xml.XmlDocument"
[xml]$myXml = $my 

It’s pipelined – every variable or the output of every function can be piped as an input to another function using the pipe (“|”) operator. It’s extremely useful when dealing with collections and other enumerable objects – allowing you to filter the elements and transform the result into whatever you need.

# a LINQ-like querying with pipelining. 
# The $_ is similar to C#'s lambda argument "x", e.g. Where(x => x.Name == "chrome") 
$ids = Get-Process | Where-Object {$_.Name -eq "chrome" } | Select-Object -Property ID 

# Where-Object and Select-Object can also be written "where" and "select": 
$ids = Get-Process | where {$_.Name -like "chrome" } | select -Property ID 
# "where" and "select" are aliases. Use Get-Alias to see all aliases in PowerShell.

It supports everything you know from .NET – although PowerShell defines its own idioms with regards to common operations, such as formatting strings for joining paths, you can use all objects from .NET to do the same thing.

$combined = Join-Path $rootDirectory "mySubdirectory" 
# can be written as: $combined = [System.IO.Path]::Combine($rootDirectory, "mySubdirectory")

It’s concise – everything in PowerShell can be verbose or very concise. By leveraging aliases there are many ways to say what you mean. Here’s an example: the following function gets a list of changed files between two SVN revisions, by executing the svn diff command, outputting the result as XML which looks like this:

<?xml version="1.0" encoding="UTF-8"?>

This XML is stored into an $output object, which is defined as a .NET XmlDocument. The object is than being queried via XPath to get only items of kind ‘file’ which are ‘modified’. Finally, only the text of the node is selected.

function Get-SvnDiff($fromRevision, $toRevision)
  [xml]$output = & svn diff -r $("{0}:{1}" -f $fromRevision, $toRevision) --summarize --xml
  $output.SelectNodes("/diff/paths/path[@item='modified' and @kind='file']") | % { $_."#text"}

Quite a lot is going on in this example, which demonstrate the Power of PowerShell:

  • A process is started with the & symbol. Can also be started with Start-Process for more options.
  • Strings can be formatted with –f, however in this case they are a part of an expression, and have to be escaped into their own expression by using $().
  • The % symbol is an alias of foreach, which is an alias of ForEach-Object. Use Get-Alias -Definition ForEach-Object to see all the alias definitions, or Get-Alias %, for example.
  • The last line in a function (or selection) is what gets returned (or printed out). In this case, the result of the last line is a single string, containing the value ‘other.txt’ (it’s the only modified file). If there was more than one result, the return would be an array.
  • Finally, to call this method, we use Get-SvnDiff 1000 2000, and we can use the pipe operator (“|”) to continue modifying the returned values.

And that’s it! Sure, there is way more to know about PowerShell, but this should be more than enough to get started, or at least, not to get lost when reading and writing simple scripts.

Happy scripting!

  • Well, slightly less now.

My top 3 favorite Windows productivity tools!

There are way more tools I love and use every day, but the following 3 are my must-haves on every Windows machine I use, so I wanted to share them with you. And they are completely free!

(Ok, there are actually 4 – the first one, Chocolatey is a pre-requisite to the others. Chocolatey is a package manager, like apt-get, but made with Windows in mind. If you haven’t already, install this first! I’ll wait.)

#1 – Rapid Environment Editor (RapidEE)

C:\> choco install Devbox-RapidEE

RapidEE is an editor for the Windows environment variables! No more being stuck squinting in the unresizeable Windows dialog, this tool will not only let you add, search and modify envvars easily, it will also show invalid paths in red, letting you find dead entries quickly!

#2 – Everything

C:\> choco install Everything

Imagine a list view, with a search box at the top, and nothing else, which finds any file in your computer immediately – you have just imagined Everything. This lightweight search engine indexes your entire hard drive(s) in seconds after installing, letting you then find any file, anywhere! My preference is to bind the shortcut for a New Window to Ctrl-Shift-Z, so I can pop open Everything everywhere with this single chord. Set it under Tools – Options – Keyboard.

#3 – Ditto

C:\> choco install ditto

Ditto is a clipboard manager for Windows. If you ever used ReSharper’s Clipboard Manager you will appreciate this little tool – it remembers every single Ctrl-C you’ve ever done, allowing you to paste back any item at any time! The default shortcut for Ditto is Ctrl-`, and it is packed with features, such as searching, exporting and removing entries, with full unicode support and ability to sync clipboards between machines. What’s more, it supports “smart paste”, allowing you to paste entries in places such as the Windows Command Prompt, which will only support proper copying and pasting only in Windows 10.

Regular Expression enhancements in ReSharper 9

Yesterday, JetBrains announced the first public EAP of ReSharper 9! While I’m sure they will dedicate whole blog posts about the new (and truly amazing!) features of ReSharper 9, I wanted to beat JetBrains to the punch, and let you know about one incredible feature (which I consider a “killer feature” of ReSharper 9). I am talking about the improvements in dealing with Regular Expressions (RegEx)!

Update: read all about the new Regex support in all its glory on the JetBrains blog!

There were always 2 ways to write regular expressions in C# code – either inside one of the methods of Regex class, e.g. Regex.IsMatch, or defining a string literal as a constant, and using it in a Regex class later.

ReSharper 9 has Syntax Highlighting and Code Completion for regex literals! When you start typing a regex into a Regex class, you’ll see a list of suggestions, which is being narrowed down as you type:

There’s also a special syntax highlighting for the complete regex – as well as automatic checking for errors:

However, the “killer feature” is the ability to validate and test the regular expressions straight from Visual Studio! Pressing Alt-Enter anywhere inside the expression will bring the Quick Actions menu, with a new option to Validate regular expression:

Which opens a new window that lets you input some text to validate your regular expression!

But the best part of these new enhancements is the ability to transform any string literal into a regular expression! Simply press Alt-Enter inside the string literal, and select Make regular expression here:

Will turn into:

And you get all the regex features (validation, intellisense, etc) inside your regex string literals!

Amazing! Smile

Debug symbols, dotPeek and long paths in Visual Studio

In my previous post, I explained how to use the symbol server in dotPeek 1.2 to debug any assembly in Visual Studio, allowing you to set breakpoints and step into any method (provided it was decompiled by dotPeek).

While this is great, I noticed that there was one particular method I couldn’t step into – the moment I tried I got the sadly familiar Source Not Found page:

Clicking the Browse and find… link did nothing, and the Source search information dropdown appeared below. Expanding it, I could see where Visual Studio attempted to load the source file from:

Locating source for 'C:\Users\Igal\AppData\Local\JetBrains\dotPeek\v1.2\SymbolCache\CSharp\Microsoft.VisualStudio.ProjectSystem.VS.Implementation.pdb\538887009A094E419882756878C69B2A1\Microsoft.VisualStudio.ProjectSystem.VS.Implementation\VisualStudio\ProjectSystem\VS\Implementation\Package\Automation\OAProjectItems.cs'. (No checksum.)
The file 'C:\Users\Igal\AppData\Local\JetBrains\dotPeek\v1.2\SymbolCache\CSharp\Microsoft.VisualStudio.ProjectSystem.VS.Implementation.pdb\538887009A094E419882756878C69B2A1\Microsoft.VisualStudio.ProjectSystem.VS.Implementation\VisualStudio\ProjectSystem\VS\Implementation\Package\Automation\OAProjectItems.cs' does not exist.
Looking in script documents for 'C:\Users\Igal\AppData\Local\JetBrains\dotPeek\v1.2\SymbolCache\CSharp\Microsoft.VisualStudio.ProjectSystem.VS.Implementation.pdb\538887009A094E419882756878C69B2A1\Microsoft.VisualStudio.ProjectSystem.VS.Implementation\VisualStudio\ProjectSystem\VS\Implementation\Package\Automation\OAProjectItems.cs'...
Looking in the projects for 'C:\Users\Igal\AppData\Local\JetBrains\dotPeek\v1.2\SymbolCache\CSharp\Microsoft.VisualStudio.ProjectSystem.VS.Implementation.pdb\538887009A094E419882756878C69B2A1\Microsoft.VisualStudio.ProjectSystem.VS.Implementation\VisualStudio\ProjectSystem\VS\Implementation\Package\Automation\OAProjectItems.cs'.
The file was not found in a project.

And so on. Quick search for OAProjectItems.cs using my most favorite tool, Everything, revealed that it was indeed present in that location, so why couldn’t Visual Studio open it? I decided to open the file manually by pasting its full path into the Start – Run dialog (Win-R), but then I got the following error:

Finally, I tried to go to the file location using the cmd, and I got my answer – the path was simply too long for Windows (and therefore, Visual Studio) to handle!

Windows has an unfortunate MAX_PATH limitation at 260 characters is the source of great pain, and I hope Microsoft fixes it one day. Meanwhile, here’s how to “work around” this particular issue: dotPeek generates debug symbols under %LOCALAPPDATA%\JetBrains\dotPeek\v1.2\SymbolCache\CSharp for C# code, and unfortunately, it isn’t possible to configure. Luckily, it is possible to configure where Visual Studio looks for debug symbols. This is a little-known page in the Solution properties called Debug Source Files:

What I did was copy the entire contents of dotPeek’s SymbolCache\CSharp directory into a local directory d:\sym, and added it to the search list (pictured above). Also, I made sure to delete everything from the bottom list (Do not look for these source files) – if Visual Studio is unable, for any reason, to open a source file, it will add it to this blacklist. It was filled with the files I needed, so I removed them from the list.

After doing this, I could go back to debugging, was was able to step into methods that were previously unavailable!

As the comments below mention, this is also possible to do without copying, by creating a (symbolic) link between the directories! From an elevator command shell, run:

mklink /D d:\sym %localappdata%\JetBrains\dotPeek\v1.2\SymbolCache\CSharp

Thanks guys!

Happy debugging!

How to debug anything with Visual Studio and JetBrains dotPeek v1.2!

Sometimes, we wish we could just step into some 3rd party library, to figure out how it works, but we either don’t have the source code, or otherwise just can’t. Fortunately, this is made possible by dotPeek v1.2 that was just released, which can act as a symbol server for decompiled assemblies!

So let’s suppose we want to put a breakpoint inside Console.WriteLine (or any other method in any other assembly). Here’s what we need to do:

  • Open dotPeek, add the required assemblies to the Assembly Explorer, and press the Start Symbol Server button.

You can configure the port and the symbol generation settings in Tools – Options. The default address is http://localhost:33417/

  • In Visual Studio, go to Tools – Options, then navigate to Debugging – Symbols. Add the location of the dotPeek symbol server.

In addition, make sure that Just My Code (in General) is unchecked, and press OK. Some symbols will be loaded, this might take a few moments.

  • Next, we need to set a breakpoint inside the method which we’re interested in. This can be done with a little-known Visual Studio trick, allowing you to create a breakpoint at any function. Go to Debug – New Breakpoint – Break at Function, and in the dialog enter the fully qualified method name, e.g. System.Console.WriteLine.

After pressing OK, you’ll get a message saying “IntelliSense could not find the specified location. Do you still want to set the breakpoint?”. It’s fine – press Yes.

  • Finally, start your application with the debugger (F5) and you will stop at the breakpoint! You can use all familiar debugging options, such as stepping over/into, watch, autos and the datatip.

Happy Debugging!

How I Improved Sharing My Blog Posts On Twitter With This One Simple Trick!

(I’m so sorry for writing this title! They made me do it, I swear! I won’t ever do it again!)

I came across a wonderful blog post the other day, titled Cassandra vs MongoDB vs CouchDB vs Redis vs Riak vs HBase vs Couchbase vs Neo4j vs Hypertable vs ElasticSearch vs Accumulo vs VoltDB vs Scalaris comparison, written by Kristof Kovacs (@kkovacs). Being the twitter addict I am, I immediately felt the need to share this post, and I was met with the eternal twitter dilemma – which part of the title/credit I omit, due to the 140 character limit. I ended up doing this:

Which is what most people would do (chop enough of the title to fit, use a URL shortener for the link, the twitter way). Which didn’t leave room to give credit to the original author of the post (a fact pointed to me by @omervk, thank you very much!)

I was wondering if there was a way to encode this information (author’s twitter handle) within the post itself, using html meta tags. Then I realized that most online tech blogs have this feature available to them – any time you paste a link from, say, Engadget or The Verge on twitter, a preview of the article is visible in the twitter stream.

Turns out, this is available not just for tech blogs, but to everyone! Enter Twitter Cards.

Twitter Cards is exactly what it sounds like – you can embed media and other information to your own tweets! There are many types of cards to choose from, but for my purposes I wanted the Summary card – a simple card including a title, description, thumbnail and the twitter account information. This requires adding various html meta tags to your page, all detailed in the URL above.

To enable Twitter Cards, you will need to Apply for the service, after adding the meta tags on your blog/site. The application process supposedly takes up to 72 hours, but in my case I was approved within minutes! Your mileage may vary.

Here is what I did to enable this for my WordPress blog:
  1. Install Twitter Cards Meta plugin
  2. After installing and activating, go to Settings – Twitter Cards Meta
  3. Fill in your information (e.g. your twitter username)
  4. Go to the Twitter Cards Validation page, and log in with your twitter account
  5. Select desired card type (I selected Summary)
  6. Click the Validate & Apply tab, and paste a link to any one of your blog posts
  7. If you’re satisfied with the results, press Go and after seeing the preview, apply for approval

It might take up to 72 hours to get approved, but usually this happens within minutes (as it was in my case). You will get an email from twitter when you’re approved:

And voilà! Now, every time someone shares a link to a blog post I wrote, it will contain a preview, and my twitter handle!

Turning on Twitter Cards will also give you much better Twitter Analytics – with much more detailed information about clicks and retweets of your cards!

Code, music and single malt whisky!