Code musings in C# – My coding journey and discovery

Merge git repositories

We have more than 20 Git repositories for the same project, but it causes lots of headaches for building. We’ve decided to merge most of them into a single one, keeping the file history when possible.

This script uses newren/git-filter-repo and as such needs Python 3.

It’s done in 3 steps:

clone the local repositories from d:\dev\xxx to d:\devnew\xxx using git-filter-repo with source and target parameters
create a new repository at d:\devnew\merged
merge from d:\devnew\xxx to d:\devnew\merged\xxx using git merge --allow-unrelated-histories

$repos = @(
    "Build",
    "Deploy",
    "External",
    "API",
    "Database",
    "Server",
    "Services",
    "UI",
    "Utils"
)

$currentLocation = "D:\Dev"
$newLocation = "D:\DevNew"

# removed:
# - check that git and python3 are installed and on the PATH
# - cleanup previous merges
# - check that repositories are clean
# - logging/write-host

$total = $repos.Length
$nb = 0

foreach ($repo in $repos) {
	$nb++
	Write-Host "Copy of $repo ($nb/$total)" -ForegroundColor Blue

	# checkout all remote branches in local
	git -C $currentLocation\$repo fetch -p
	$remoteBranches = (git -C $currentLocation\$repo branch -l -r).Split("`n").Trim().Replace("origin/","") | where { -not $_.StartsWith("HEAD ->") }
	$localBranches = (git -C $currentLocation\$repo branch -l).Split("`n").Replace("*","").Trim()
	foreach ($remoteBranch in $remoteBranches) {
		if (-not ($remoteBranch -in $localBranches)) {	
			git -C $currentLocation\$repo branch $remoteBranch "origin/$remoteBranch"
		}
		else {
			git -C $currentLocation\$repo rebase "origin/$remoteBranch" $remoteBranch
		}
	}

	# re-checkout dev
	git -C $currentLocation\$repo checkout dev
	git -C $currentLocation\$repo fetch
	git -C $currentLocation\$repo rebase
	
	# https://stackoverflow.com/a/14728706/
	git -C $currentLocation\$repo -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now gc

	# initialize new repo
	git init $newLocation\$repo
	
	# define ignored files and folders in history (old data/tests/mistakes you won't ever need again)
	$ignores = @()
	elseif ($repo -eq "Build") {
		$ignores = @("installers", "xunit", "opencover", "reportgenerator")
	}
	elseif ($repo -eq "Server") {
		$ignores = @("TestProject", "OldProject")
	}
	
	# build filter string
	$filterExpression = "python git-filter-repo --source $currentLocation\$repo --target $newLocation\$repo --path-rename :$repo/ --path packages --path trunk --path-glob *.suo"
	foreach ($ignore in $ignores) {
		if ($ignore.IndexOf("*") -gt -1) {
			$filterExpression += " --path-glob $ignore"
		}
		else {
			$filterExpression += " --path $ignore"
		}
	}
	
	# move tags/* and branches/* to $repo/tags/* and $repo/branches/*
	$filterExpression += " --invert-paths --mailmap $PSScriptRoot\mailmap.txt --refname-callback 'return refname.replace(`"refs/heads/`", `"refs/heads/$repo/`")replace(`"refs/tags/`", `"refs/tags/$repo/`")'"
	
	# copy the git history while filtering the repo
	Invoke-Expression $filterExpression
}

# create merged repo
git init $newLocation\Merged
git -C $newLocation\Merged checkout -b dev

# initialize LFS for binary files
git -C $newLocation\Merged lfs install
git -C $newLocation\Merged lfs track "*.dll"
git -C $newLocation\Merged lfs track "*.exe"

# operations on new repositories
$nb = 0
foreach ($repo in $repos) {
	$nb++
	Write-Host "Starts merging $repo into $newLocation\Merged ($nb/$total)" -ForegroundColor Blue

	# get list of imported branches and tags
	$branches = (git -C $newLocation\$repo branch -l).Split("`n").Replace("*","").Trim() | where { $_.StartsWith($repo) }
	$tags = $(git -C $newLocation\$repo tag -l).Split()

	# add remote
	git -C $newLocation\Merged remote add "$($repo)_remote" $newLocation\$repo
	git -C $newLocation\Merged fetch "$($repo)_remote"
	
	# create branches
	foreach ($branch in $branches) {
		git -C $newLocation\Merged branch $branch "$($repo)_remote/$branch"
	}
	
	# merge imported repo into merged repo
	git -C $newLocation\Merged checkout dev
	git -C $newLocation\Merged merge --allow-unrelated-histories "$repo/dev"

	git -C $newLocation\Merged remote remove "$($repo)_remote"
}

git -C $newLocation\Merged -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now gc

$EndDate = Get-Date

$duration = New-Timespan -Start $StartDate -End $EndDate
Write-Host "====================== DONE IN $($duration.minutes) mn ======================" -ForegroundColor Green

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

$repos = @(

"Build",

"Deploy",

"External",

"API",

"Database",

"Server",

"Services",

"UI",

"Utils"

)

$currentLocation = "D:\Dev"

$newLocation = "D:\DevNew"

# removed:

# - check that git and python3 are installed and on the PATH

# - cleanup previous merges

# - check that repositories are clean

# - logging/write-host

$total = $repos.Length

$nb = 0

foreach ($repo in $repos) {

$nb++

Write-Host "Copy of $repo ($nb/$total)" -ForegroundColor Blue

# checkout all remote branches in local

git -C $currentLocation\$repo fetch -p

$remoteBranches = (git -C $currentLocation\$repo branch -l -r).Split("`n").Trim().Replace("origin/","") | where { -not $_.StartsWith("HEAD ->") }

$localBranches = (git -C $currentLocation\$repo branch -l).Split("`n").Replace("*","").Trim()

foreach ($remoteBranch in $remoteBranches) {

if (-not ($remoteBranch -in $localBranches)) {

git -C $currentLocation\$repo branch $remoteBranch "origin/$remoteBranch"

}

else {

git -C $currentLocation\$repo rebase "origin/$remoteBranch" $remoteBranch

}

# re-checkout dev

git -C $currentLocation\$repo checkout dev

git -C $currentLocation\$repo fetch

git -C $currentLocation\$repo rebase

# https://stackoverflow.com/a/14728706/

git -C $currentLocation\$repo -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now gc

# initialize new repo

git init $newLocation\$repo

# define ignored files and folders in history (old data/tests/mistakes you won't ever need again)

$ignores = @()

elseif ($repo -eq "Build") {

$ignores = @("installers", "xunit", "opencover", "reportgenerator")

}

elseif ($repo -eq "Server") {

$ignores = @("TestProject", "OldProject")

}

# build filter string

$filterExpression = "python git-filter-repo --source $currentLocation\$repo --target $newLocation\$repo --path-rename :$repo/ --path packages --path trunk --path-glob *.suo"

foreach ($ignore in $ignores) {

if ($ignore.IndexOf("*") -gt -1) {

$filterExpression += " --path-glob $ignore"

}

else {

$filterExpression += " --path $ignore"

}

# move tags/* and branches/* to $repo/tags/* and $repo/branches/*

$filterExpression += " --invert-paths --mailmap $PSScriptRoot\mailmap.txt --refname-callback 'return refname.replace(`"refs/heads/`", `"refs/heads/$repo/`")replace(`"refs/tags/`", `"refs/tags/$repo/`")'"

# copy the git history while filtering the repo

Invoke-Expression $filterExpression

}

# create merged repo

git init $newLocation\Merged

git -C $newLocation\Merged checkout -b dev

# initialize LFS for binary files

git -C $newLocation\Merged lfs install

git -C $newLocation\Merged lfs track "*.dll"

git -C $newLocation\Merged lfs track "*.exe"

# operations on new repositories

$nb = 0

foreach ($repo in $repos) {

$nb++

Write-Host "Starts merging $repo into $newLocation\Merged ($nb/$total)" -ForegroundColor Blue

# get list of imported branches and tags

$branches = (git -C $newLocation\$repo branch -l).Split("`n").Replace("*","").Trim() | where { $_.StartsWith($repo) }

$tags = $(git -C $newLocation\$repo tag -l).Split()

# add remote

git -C $newLocation\Merged remote add "$($repo)_remote" $newLocation\$repo

git -C $newLocation\Merged fetch "$($repo)_remote"

# create branches

foreach ($branch in $branches) {

git -C $newLocation\Merged branch $branch "$($repo)_remote/$branch"

}

# merge imported repo into merged repo

git -C $newLocation\Merged checkout dev

git -C $newLocation\Merged merge --allow-unrelated-histories "$repo/dev"

git -C $newLocation\Merged remote remove "$($repo)_remote"

}

git -C $newLocation\Merged -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now gc

$EndDate = Get-Date

$duration = New-Timespan -Start $StartDate -End $EndDate

Write-Host "====================== DONE IN $($duration.minutes) mn ======================" -ForegroundColor Green

I’ve removed a lot of code for brievity, so maybe it won’t work out of the box, but you should get the general idea.

Next up: pushing the new repository, then migrate the developers workstations, for which I wrote another script:

# removed: checks and logs

$oldRootPath = "D:\Dev\"
$newRootPath = "D:\Dev\Mezzo\"

Install-Module -Name 'IISAdministration' -Force -ErrorAction Stop
Import-Module IISAdministration

# get sites list
$sites = @()
Get-IISSite | % {
	$sitepath = $_.Applications.VirtualDirectories.PhysicalPath

	if ($sitepath -is [array]) {
		$sitepath = $sitepath[0]
	}

	$sites += @{
		PhysicalPath = $sitepath
		Name = $_.Name
	}
}

# iterate over sites
foreach ($site in $sites) {
	if (-not $site.PhysicalPath.ToLower().StartsWith($oldRootPath.Tolower())) {
		continue
	}
	
	$sitename = $site.Name
	$sitepath = $site.PhysicalPath

	$relativePhysicalPath = $sitepath.Substring($oldRootPath.Length)
	$skip = $false
	$virtualDirectoryNames = @()
	switch ($relativePhysicalPath.ToLower()) {
		"ui\users" {
			$virtualDirectoryNames = @("js")
		}
		"ui\admin" {
			$virtualDirectoryNames = @("js")
		}
	}
	
	$newPhysicalPath = "$newRootPath$relativePhysicalPath"

	$manager = Get-IISServerManager
	$rootsite = $manager.Sites[$sitename].Applications["/"].VirtualDirectories["/"]
	$rootsite.PhysicalPath = $newPhysicalPath
	
	foreach ($vdirname in $virtualDirectoryNames) {
		$vdirname = "/" + ($vdirname -replace "\\", "/")

		$vdir = $manager.Sites[$sitename].Applications["/"].VirtualDirectories[$vdirname]
		if ($vdir -eq $null) {
			continue
		}

		$relativePhysicalPath = $vdir.Attributes["physicalPath"].Value.Substring($oldRootPath.Length)
		$newPhysicalPath = "$newRootPath$relativePhysicalPath"
		$vdir.PhysicalPath = $newPhysicalPath
	}
	
	$manager.CommitChanges()
}

# restart IIS
& iisreset

# removed: checks and logs

$oldRootPath = "D:\Dev\"

$newRootPath = "D:\Dev\Mezzo\"

Install-Module -Name 'IISAdministration' -Force -ErrorAction Stop

Import-Module IISAdministration

# get sites list

$sites = @()

Get-IISSite | % {

$sitepath = $_.Applications.VirtualDirectories.PhysicalPath

if ($sitepath -is [array]) {

$sitepath = $sitepath[0]

}

$sites += @{

PhysicalPath = $sitepath

Name = $_.Name

}

# iterate over sites

foreach ($site in $sites) {

if (-not $site.PhysicalPath.ToLower().StartsWith($oldRootPath.Tolower())) {

continue

}

$sitename = $site.Name

$sitepath = $site.PhysicalPath

$relativePhysicalPath = $sitepath.Substring($oldRootPath.Length)

$skip = $false

$virtualDirectoryNames = @()

switch ($relativePhysicalPath.ToLower()) {

"ui\users" {

$virtualDirectoryNames = @("js")

}

"ui\admin" {

$virtualDirectoryNames = @("js")

}

$newPhysicalPath = "$newRootPath$relativePhysicalPath"

$manager = Get-IISServerManager

$rootsite = $manager.Sites[$sitename].Applications["/"].VirtualDirectories["/"]

$rootsite.PhysicalPath = $newPhysicalPath

foreach ($vdirname in $virtualDirectoryNames) {

$vdirname = "/" + ($vdirname -replace "\\", "/")

$vdir = $manager.Sites[$sitename].Applications["/"].VirtualDirectories[$vdirname]

if ($vdir -eq $null) {

continue

}

$relativePhysicalPath = $vdir.Attributes["physicalPath"].Value.Substring($oldRootPath.Length)

$newPhysicalPath = "$newRootPath$relativePhysicalPath"

$vdir.PhysicalPath = $newPhysicalPath

}

$manager.CommitChanges()

}

# restart IIS

& iisreset

Next up: migrating Jenkins jobs. With more than 170 jobs, doing it by hand is a real chore. Fortunately, you can also automate it.

# removed: variables, checks, etc

# - generate an API token in your Jenkins profile
# - create a credentials.txt file, write your login like: `login:tokenapi`
# - call: certutil -encode credentials.txt credentials.asc
# - remove the lines `BEGIN CERTIFICATE` and `END CERTIFICATE`
$credFile = Get-Content ".\credentials.asc"
if ($credFile -contains "-----BEGIN CERTIFICATE-----") {
	Write-Host "Remove the 'BEGIN CERTIFICATE' and 'END CERTIFICATE' lines from credentials.asc"
	Exit 1
}

$Credentials = $credFile.Trim()
$authHeader = @{Authorization = ('Basic ' + $Credentials) }

# list projects
$projects = Invoke-RestMethod -Method "GET" -Uri "$Url/api/json" -Headers $authHeader -ContentType "application/json; charset=utf-8"

# for each project, get the config.xml file and save it here
foreach ($job in $projects.jobs) {
    $name = $job.name
    $configUrl = "$Url/job/$name/config.xml"

    Invoke-RestMethod -Method "GET" -Uri $configUrl -Headers $authHeader -ContentType "text/xml" | Out-File "jobs\$name.xml"
}

# removed: variables, checks, etc

# - generate an API token in your Jenkins profile

# - create a credentials.txt file, write your login like: `login:tokenapi`

# - call: certutil -encode credentials.txt credentials.asc

# - remove the lines `BEGIN CERTIFICATE` and `END CERTIFICATE`

$credFile = Get-Content ".\credentials.asc"

if ($credFile -contains "-----BEGIN CERTIFICATE-----") {

Write-Host "Remove the 'BEGIN CERTIFICATE' and 'END CERTIFICATE' lines from credentials.asc"

Exit 1

}

$Credentials = $credFile.Trim()

$authHeader = @{Authorization = ('Basic ' + $Credentials) }

# list projects

$projects = Invoke-RestMethod -Method "GET" -Uri "$Url/api/json" -Headers $authHeader -ContentType "application/json; charset=utf-8"

# for each project, get the config.xml file and save it here

foreach ($job in $projects.jobs) {

$name = $job.name

$configUrl = "$Url/job/$name/config.xml"

Invoke-RestMethod -Method "GET" -Uri $configUrl -Headers $authHeader -ContentType "text/xml" | Out-File "jobs\$name.xml"

}

Now we can version the Jenkins configs. Then we edit, and update them.

# removed: variables, checks, etc

$credFile = Get-Content ".\credentials.asc"
if ($credFile -contains "-----BEGIN CERTIFICATE-----") {
	Write-Host "Remove the 'BEGIN CERTIFICATE' and 'END CERTIFICATE' lines from credentials.asc"
	Exit 1
}

$Credentials = $credFile.Trim()
$authHeader = @{Authorization = ('Basic ' + $Credentials) }

$configUrl = "$Url/job/$JobName/config.xml"

# push $localConfigFile to $Url
Invoke-RestMethod -Method "POST" -Uri $configUrl -Headers $authHeader -ContentType "application/xml" -InFile $localConfigFile

# removed: variables, checks, etc

$credFile = Get-Content ".\credentials.asc"

if ($credFile -contains "-----BEGIN CERTIFICATE-----") {

Write-Host "Remove the 'BEGIN CERTIFICATE' and 'END CERTIFICATE' lines from credentials.asc"

Exit 1

}

$Credentials = $credFile.Trim()

$authHeader = @{Authorization = ('Basic ' + $Credentials) }

$configUrl = "$Url/job/$JobName/config.xml"

# push $localConfigFile to $Url

Invoke-RestMethod -Method "POST" -Uri $configUrl -Headers $authHeader -ContentType "application/xml" -InFile $localConfigFile

Migrate YUI components

I work on an application that uses quite a lot of Javascript components based on the YUI framework, which has been obsolete since around 2014. We recently had the budget to start migrating them to plain Javascript.

How YUI widgets work

Our YUI components use the syntax of YUI3. Sometimes they also use widgets from YUI2 : these have to be rebuilt, or migrated to use another library.

A YUI 3 component is structured as follows:

Unable to type text in inputs in IE 11, missing caret

The app I’m working on allows a user working on a document to open a pop-in (Bootstrap popup), select something, validate, and it adds a field on the document, where the user can type something. After working for some time on the widget and changing many things, I test in IE11 and find myself unable to type anything anywhere, after selecting a value. The text input fields are focused (they have the Bootstrap blue halo), but the caret does not appear, and typing doesn’t do anything. Clearing another field that already has some value (using the “X” in the field) makes the inputs work again. All other browsers are working fine. It smells very strongly like a IE bug. A very similar bug is already documented here but it looks like it should have been fixed in 2014, and I’m using Windows 10 (so one would assume it has this fix). Anyway, the bug points towards a focused element being removed from the DOM. I have tried to just focus something else and then blur it, but it didn’t work. My salvation came when I realized that the content of the popup was inside an iframe. Selecting and bluring the iframe’s body did the trick :

$('#modalWindow').find('iframe').contents().find('body').focus().blur()

1	$('#modalWindow').find('iframe').contents().find('body').focus().blur()

Always run Visual Studio as administrator on Windows 10

You might be working with a program that always require administrator privileges, such as a service listening on a port. Up to Windows 10, you just right clicked the devenv exe, and checked “always run as administrator” in the compatibility tab. But in Windows 10, the compatibility tab is missing for the Visual Studio exe. Why? No idea. But here’s how to fix it.

To just run the shortcut as admin : right click on the sortcut > in the “properties” tab, click “advanced”, then check “always run as administrator”. Simple, fast, easy.

If you want to run Visual Studio as administrator also when opening a .sln or .csproj file, it’s not so easy. Here are the logical, coherent, and easy to find steps to fix this: (labels translated to english from french, sorry if it doesn’t match)

Open the start menu, type “troubleshoot”, and open “check your computer state and solve problems”
On the bottom left, click “troubleshoot compatibility problems”
Click on “advanced” on the left then “run as administrator”
Click again on “advanced”, then uncheck “automatically fix”, then “next”
Search for Visual Studio in the list, then “next”
Make sure “fix the program” is checked, then “next”
Select “troubleshoot the program” (bottom option)
Check “the program requires additional rights”, then “next”
Click “test the programe” then “next”
Select “save the parameters for the program” then “close”

Easy, logical, simple. Thanks Microsoft!

Git-svn and branches

I’m using git-svn to leverage the power of git while the rest of the company prefers the simplicity of svn.

I have created a local branch, intending to merge it in the trunk/master when I’m done, but it turns out it’s much more complicated than expected. So, I want to commit it on svn.
I already have checked out the full repository.

The most simple solution I have found; all other solutions fiddle with the git config files, and I don’t like it:

Checkout the branch: git checkout mybranch
Rename the local branch: git branch -m mybranch-temp
Create a remote branch “mybranch” in SVN using Tortoise SVN
Fetch the SVN repository to get the new branch: git svn fetch
Check that the new SVN branch has been fetched: git branch -r (or git branch -avv )
Get back to the master branch: git checkout master
Checkout the remote branch: git checkout -b mybranch-svn origin/mybranch
Rebase my local branch into the local svn branch: git rebase mybranch-temp
Commit the branch in svn: git svn dcommit ; use --dry-run to check the branch it will commit to, because sometimes it still commits to the trunk (I haven’t found out why).

Unit testing MongoDB queries

When writing unit tests, it’s common to stop at the database access layer.
You might have a “dumb” data access layer that just passes stored procedure names and parameters to the SQL database, for instance.
It’s usually very hard to test this layer, since it requires either that your build server has access to a SQL Server instance, or that a SQL Server is running on the build server.
Plus, we’re getting out of unit tests and are entering integration tests, here.

Using a more advanced tool like Entity Framework, in order to test your complex EF queries, there are usually methods to insert fake data into a fake container, like Test Doubles, InMemory, or Effort.

Using MongoDB, you might encounter the same problem : how do I test that my complex queries are working ?

Here for instance, I get the possible colors of a product; how do I know it works using unit tests?

public IEnumerable<Color> GetColors(string productId) {
	var coll = db.GetCollection<BsonDocument>("products");
	var aggregate = coll.Aggregate(new AggregateOptions { AllowDiskUse = true })
		.Match(new BsonDocument { { "ProductId", productId }, { "Color", new BsonDocument("$ne", "") } })
		.Unwind(new StringFieldDefinition<BsonDocument>("Color"))
		.Group(new BsonDocument { { "_id", "$Color" }, { "Count", new BsonDocument("$sum", 1) } })
		.Sort(Builders<BsonDocument>.Sort.Descending("Count"))
		.Limit(100);

	return aggregate
		.ToList()
		.Select(doc => new Color { ColorValue = doc["_id"].AsString, Count = doc["Count"].AsInt32 });
}

public IEnumerable<Color> GetColors(string productId) {

var coll = db.GetCollection<BsonDocument>("products");

var aggregate = coll.Aggregate(new AggregateOptions { AllowDiskUse = true })

.Match(new BsonDocument { { "ProductId", productId }, { "Color", new BsonDocument("$ne", "") } })

.Unwind(new StringFieldDefinition<BsonDocument>("Color"))

.Group(new BsonDocument { { "_id", "$Color" }, { "Count", new BsonDocument("$sum", 1) } })

.Sort(Builders<BsonDocument>.Sort.Descending("Count"))

.Limit(100);

return aggregate

.ToList()

.Select(doc => new Color { ColorValue = doc["_id"].AsString, Count = doc["Count"].AsInt32 });

}

MongoDB has an “inMemory” storage engine, but it’s reserved to the (paid) Enterprise edition. Fortunately, since 3.2, even the Community edition has a not-very-well-documented “ephemeralForTests” storage engine, which loads up an in-memory Mongo instance, and does not store anything on the hard drive (but has poor performances). Exactly what we need!

Before running the data access layer tests, we will need to fire up an in-memory instance of MongoDB.
This instance will be common to all the tests for the layer, otherwise the test runner will fire up new tests faster than the system releases resources (file and ports locks).

You will have to extract the MongoDB binaries in your sources repository somewhere, and copy them besides your binaries on build.

The following wrapper provides a “Query” method that allows us to access the Mongo instance through command-line, bypassing the data access layer, in order to insert test data or query insertion results.

public class MongoDaemon : IDisposable {
	public const string ConnexionString = "mongodb://localhost:27017";
	public const string DbName = "test";
	public const string Host = "localhost";
	public const string Port = "27017";
	private readonly string assemblyFolder;
	private readonly string dbFolder;
	private readonly string mongoFolder;
	private Process process;

	public MongoDaemon() {
		this.assemblyFolder = Path.GetDirectoryName(new Uri(typeof(MongoDaemon).Assembly.CodeBase).LocalPath);
		this.mongoFolder = Path.Combine(this.assemblyFolder, "mongo");
		this.dbFolder = Path.Combine(this.mongoFolder, "temp");

		// re-create db folder if it exists
		if (Directory.Exists(this.dbFolder)) {
			Directory.Delete(this.dbFolder, true);
			Directory.CreateDirectory(this.dbFolder);
		}

		this.process = new Process();
		process.StartInfo.FileName = Path.Combine(this.mongoFolder, "mongod.exe");
		process.StartInfo.Arguments = "--dbpath " + this.dbFolder + " --storageEngine ephemeralForTest";
		process.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
		process.Start();
	}

	public void Dispose() {
		Dispose(true);
		GC.SuppressFinalize(this);
	}

	public string Query(string query) {
		var output = string.Empty;

		var procQuery = new Process {
			StartInfo = new ProcessStartInfo {
				FileName = Path.Combine(this.mongoFolder, "mongo.exe"),
				Arguments = string.Format("--host {0} --port {1} --quiet --eval \"{2}\"", Host, Port, query),
				UseShellExecute = false,
				RedirectStandardOutput = true,
				CreateNoWindow = true
			},
		};

		procQuery.Start();

		// read query output
		while (!procQuery.StandardOutput.EndOfStream) {
			output += procQuery.StandardOutput.ReadLine() + "\n";
		}

		// wait 2 seconds max before killing it
		if (!procQuery.WaitForExit(2000)) {
			procQuery.Kill();
		}

		return output;
	}

	protected virtual void Dispose(bool disposing) {
		if (disposing) {
			// dispose managed resources
		}

		if (process != null && !process.HasExited) {
			process.Kill();
		}
	}
}

public class MongoDaemon : IDisposable {

public const string ConnexionString = "mongodb://localhost:27017";

public const string DbName = "test";

public const string Host = "localhost";

public const string Port = "27017";

private readonly string assemblyFolder;

private readonly string dbFolder;

private readonly string mongoFolder;

private Process process;

public MongoDaemon() {

this.assemblyFolder = Path.GetDirectoryName(new Uri(typeof(MongoDaemon).Assembly.CodeBase).LocalPath);

this.mongoFolder = Path.Combine(this.assemblyFolder, "mongo");

this.dbFolder = Path.Combine(this.mongoFolder, "temp");

// re-create db folder if it exists

if (Directory.Exists(this.dbFolder)) {

Directory.Delete(this.dbFolder, true);

Directory.CreateDirectory(this.dbFolder);

}

this.process = new Process();

process.StartInfo.FileName = Path.Combine(this.mongoFolder, "mongod.exe");

process.StartInfo.Arguments = "--dbpath " + this.dbFolder + " --storageEngine ephemeralForTest";

process.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;

process.Start();

}

public void Dispose() {

Dispose(true);

GC.SuppressFinalize(this);

}

public string Query(string query) {

var output = string.Empty;

var procQuery = new Process {

StartInfo = new ProcessStartInfo {

FileName = Path.Combine(this.mongoFolder, "mongo.exe"),

Arguments = string.Format("--host {0} --port {1} --quiet --eval \"{2}\"", Host, Port, query),

UseShellExecute = false,

RedirectStandardOutput = true,

CreateNoWindow = true

};

procQuery.Start();

// read query output

while (!procQuery.StandardOutput.EndOfStream) {

output += procQuery.StandardOutput.ReadLine() + "\n";

}

// wait 2 seconds max before killing it

if (!procQuery.WaitForExit(2000)) {

procQuery.Kill();

}

return output;

}

protected virtual void Dispose(bool disposing) {

if (disposing) {

// dispose managed resources

}

if (process != null && !process.HasExited) {

process.Kill();

}

We’re using the IClassFixture interface of xUnit to fire up a MongoDaemon instance that will be common to all our tests using it.
It means we need to clean up previously inserted test data at each run.

public class MongoDataStoreTests : IClassFixture<MongoDaemon> {
	private readonly MongoDaemon daemon;
	private readonly DataStore sut;

	public MongoDataStoreTests(MongoDaemon daemon) {
		this.daemon = daemon;
		this.sut = new DataStore("localhost", 27017);

		// cleanup previous tests data
		this.daemon.Query("db.products.drop()");
	}

	[Fact]
	public void Colors_are_counted() {
		// arrange : insert test data
		var id = "test";
		var nb = 5;
		for (int i = 0; i < nb; i++) {
			var color = string.Format("{0}{0}{0}", i, i, i);
			var data = "{ ProductId: '" + id + "', Color: '" + color + "' }";
			this.daemon.Query("db.products.insertOne(" + data + ")");
		}

		// act
		var result = this.sut.GetColors(id);

		// assert
		result.Should().HaveCount(nb);
	}
}

public class MongoDataStoreTests : IClassFixture<MongoDaemon> {

private readonly MongoDaemon daemon;

private readonly DataStore sut;

public MongoDataStoreTests(MongoDaemon daemon) {

this.daemon = daemon;

this.sut = new DataStore("localhost", 27017);

// cleanup previous tests data

this.daemon.Query("db.products.drop()");

}

[Fact]

public void Colors_are_counted() {

// arrange : insert test data

var id = "test";

var nb = 5;

for (int i = 0; i < nb; i++) {

var color = string.Format("{0}{0}{0}", i, i, i);

var data = "{ ProductId: '" + id + "', Color: '" + color + "' }";

this.daemon.Query("db.products.insertOne(" + data + ")");

}

// act

var result = this.sut.GetColors(id);

// assert

result.Should().HaveCount(nb);

}

There you have it: a kind-of-easy way to test your Mongo data access layer.

Nested sets references

Just bookmarking a few things here.

The “base” reference to understand hierarchical sets vs nested sets
A few practical examples ; there is a part 2 and a GitHub
A neat query to convert a traditional hierarchy (with a parent id) to a nested set ; found from this post which unfortunately uses the HierarchyID of SQL2008
Speaking of which, there is a HierarchyID data type in SQL2008. Neat, but makes it (even) harder to switch to another DB.
In a more front-oriented way, here is a javascript library to handle drag & drop of items in a nested list. It’s based on JQueryUI, but it’s pretty good.

Modify copy/paste format depending on target

Sometimes you want to copy tabular data, and paste it differently depending on the target. For instance, you want CSV when you paste in your text editor, and HTML when you paste it in an email.

Fortunately, some nice fellow has created a helper class to handle all that for you. And it’s even available on GitHub!

To sum it up, the Clipboard class has a SetDataObject method. A DataObject can get multiple contents (with different DataFormats) through multiple calls of the SetData(string, object) method.
So basically, you should create a DataObject, then call SetData once with the HTML, and once with text. The trick is that the HTML needs special formatting to work.

Start a WPF application in the notification area with Caliburn.Micro

Sometimes you’re writing a quick utility app that you wish to start directly in the notification area (near the clock), and not display anything on startup.
It’s actually very simple using Caliburn.Micro.

You probably already have customized your application bootstrapper already. All you have to do is modify the OnStartup method from DisplayRootViewFor<T> to displaying a custom tooltip. Here I’m using the Hardcodet.NotifyIcon.Wpf Nuget package:

protected override void OnStartup(object sender, StartupEventArgs e)
{
	TaskbarIcon tbi = new TaskbarIcon();
	tbi.Icon = Properties.Resources.sql_icon;
	tbi.ToolTipText = Properties.Resources.ToolTip;
}

protected override void OnStartup(object sender, StartupEventArgs e)

{

TaskbarIcon tbi = new TaskbarIcon();

tbi.Icon = Properties.Resources.sql_icon;

tbi.ToolTipText = Properties.Resources.ToolTip;

}

It seems obvious in retrospect, but it took me a while to find this, because I don’t modify the app bootstrapper often, so I forgot about this method.

Don’t forget to add behavior to this icon! Double-click, context menu…

I have created a Caliburn.Micro + MahApps template app if you wish a starting point : https://github.com/cosmo0/Caliburn.MahApps.Metro.Template

Mapping DataTables to domain objects

It’s always a pain to work with DataTables and DataSets. You’re always typing these pesky column names, it doesn’t necessarily represents the actual data model (if there are nested data), there is no type safety until runtime, it’s pretty hard to use Linq on your results… the list goes on.

Fortunately, there is a simple solution: AutoMapper. It’s very simple to cast a DataTable to a list of objects:

public class Person {
    public int Id { get; set; }
    public string Name { get; set; }
    public int Age { get; set; }
}

List<Person> people = AutoMapper.Mapper.DynamicMap<IDataReader, List<Person>>(
    sourceDataTable.CreateDataReader());

public class Person {

public int Id { get; set; }

public string Name { get; set; }

public int Age { get; set; }

}

List<Person> people = AutoMapper.Mapper.DynamicMap<IDataReader, List<Person>>(

sourceDataTable.CreateDataReader());