March 2015 – Code musings in C#

C, C++ and C++/CLI cheat sheet

No explanation of the basic concepts, just a memory help when you don’t use C/C++ frequently, or if you’re a beginner with some bases.

First question: why pointers? Answers: because functions arguments are passed by value in C and C++. Because arrays and strings are crazy in C.

How to read pointers

Here is a handy way of remembering how pointers syntax works.

In variable declaration

Go from the right to the left of the declaration, from the innermost parenthesis, going back to the right when hitting “)”

Read the name of the declaration
* is “a pointer to”
[] is “an array of”
() is “a function returning”
Finally, read the type

int *p[]; reads “p is an array of pointers to int”.
int *(*foo())(); reads “foo is a function returning a pointer to a function returning a pointer to int”.

In variable usage

Variable usage reads from left to right.

Determine “get” (read variable) or “set” (write variable)
* is “the value pointed to by”
& is “the address of”
[i] is “the element at position i”
Read the variable name

*p = 5;  reads “set the value pointed to by p”.
x = &i;  reads “get the address of i”.
printf("%i", *foo[0]);  reads “get the value pointed to by the element at position 0 of foo”.

C pointers

Manipulating a variable through a pointer is called dereferencing. It is used through an indirection expression.
Manipulating pointers through variables is called decaying.

Given an integer:

int i = 42;

1	int i = 42;

Create an pointer:

int* p = &i; // or int *p = &i; // preferred syntax

1
2
3

int* p = &i;
// or
int *p = &i; // preferred syntax
Get the memory location of the variable:

printf("%i", p); // or printf("%i", &i); // or pritf("%i", &*p); // get the address of the value pointed to by p

1
2
3
4
5

printf("%i", p);
// or
printf("%i", &i);
// or
pritf("%i", &*p); // get the address of the value pointed to by p
Get the value of the variable:

printf("%i", i); // or printf("%i", *p); // or printf("%i", *&*&*&*p); // if you want to be stupid

1
2
3
4
5

printf("%i", i);
// or
printf("%i", *p);
// or
printf("%i", *&*&*&*p); // if you want to be stupid
Set the value of the variable:

i = 42; // or *p = 42;

1
2
3

i = 42;
// or
*p = 42;
Setting a pointer to null allows to assign it to a variable later:

int *p = NULL; if (p) { // use *p }

1
2
3
4

int *p = NULL;
if (p) {
// use *p
}

C arrays

Arrays mostly can’t be manipulated directly, but rather through pointers.
The variable array is, for most intents and purposes, a pointer to the first element of the array (until it’s incremented).

Given an array of integers:

int array[] = { 45, 67, 89 };

1	int array[] = { 45, 67, 89 };

This doesn’t really creates an array. It creates 3 integers side-by-side, and stores the memory location of the first one.
array == &array == &array[0]
When you pass an array as an argument to a function, you really pass a pointer to the array’s first element, because the array decays to a pointer.
Incrementing a pointer to an array really moves the pointer to the next element of the array (the pointer is incremented by the size of the type of the pointer).

array++; // now the "array" pointer points to the second element.

1

array++; // now the "array" pointer points to the second element.
The subscript operator (the [] in array[0]) has nothing to do with arrays themselves, you can also use it on a pointer:

int *ar_p = array; printf("%i", ar_p[0]); // outputs the memory location of the first element printf("%i", *ar_p[0]); // outputs the value of the first element

1
2
3

int *ar_p = array;
printf("%i", ar_p[0]); // outputs the memory location of the first element
printf("%i", *ar_p[0]); // outputs the value of the first element

C structures

Given a structure:

struct foo {
  char name[10];
  int result;
};

struct foo f; // note you have to specify "struct" in the declaration

struct foo {

char name[10];

int result;

};

struct foo f; // note you have to specify "struct" in the declaration

Accessing members of the structure through the variable:

f.result = 42; printf("%i", f.result); // outputs "42"

1
2

f.result = 42;
printf("%i", f.result); // outputs "42"
Accessing the members through a pointer:

struct foo *p = f; (*p).result = 42; // or p->result = 42; // using a pointer-to-member operator

1
2
3
4

struct foo *p = f;
(*p).result = 42;
// or
p->result = 42; // using a pointer-to-member operator

C++ references

C++ is much less in love with pointers than C, and prefers using reference variables. They are mostly used in the same way as pointers, but their address can’t change (they always points to the same variable).
The main usage of a reference variable is for function parameters, to pass the variables by reference (by default it’s by value), which allows to not use pointers.

Given an integer:

int i = 42;

1	int i = 42;

Create a reference to a variable with int &r = i; ; this is called binding a reference to an object
You do not need to dereference a reference to access the variable’s value:

i = 42; // or int &r = i; r = 42; // should generate about the same machine code as int *p = i; *p = 42;

1
2
3
4
5
6
7
8

i = 42;
// or
int &r = i;
r = 42;

// should generate about the same machine code as
int *p = i;
*p = 42;
To make sure the arguments are passed by reference in a method:

void foo(int &bar) { bar = 42; } // call it with the variable itself int i = 0; foo(i); printf("%i", i); // outputs 42

1
2
3
4
5
6
7
8

void foo(int &bar) {
bar = 42;
}

// call it with the variable itself
int i = 0;
foo(i);
printf("%i", i); // outputs 42
On the other hand, the same method using pointers must be passed the variables by reference in the call:

void foo(int *bar) { *bar = 42; } // call it with the references to the variable foo(&i);

1
2
3
4
5
6

void foo(int *bar) {
*bar = 42;
}

// call it with the references to the variable
foo(&i);

Managed instances

Instances of managed objects should be used through handles ( ^ , also called “hat”). It can be read “is a handle to the instance of the managed class”. Then, you use the handle as if it was the actual instance.

Note that, even though you don’t have to use managed objects in C++/CLI, you have to declare instances of managed objects with a handle.

public ref class Foo {
  int result; // native member
  System::String^ name; // managed member
};

// while the convention for pointers is to prefix "*" to the pointer name,
// the convention for handles is to suffix "^" to the class name
Foo^ f = gcnew Foo();
printf("%s", f.name);

public ref class Foo {

int result; // native member

System::String^ name; // managed member

};

// while the convention for pointers is to prefix "*" to the pointer name,

// the convention for handles is to suffix "^" to the class name

Foo^ f = gcnew Foo();

printf("%s", f.name);

Managed pointers

Because of garbage collection (which changes memory addresses), you can’t just use a regular C/C++ pointer to a managed objects. The managed pointers are called interior pointers. A good overview can be found here.

The syntax for interior pointers is very… declarative:

interior_ptr<int> p = &f->result;

1	interior_ptr<int> p = &f->result;

Native pointers are automatically converted to interior pointers (the inverse is not true):

void foo(interior_ptr<int> i) {
  *i = 42;
}

foo(&f->result); // passing a native pointer to the method

void foo(interior_ptr<int> i) {

*i = 42;

}

foo(&f->result); // passing a native pointer to the method

As with pointers in C++, usage interior pointers is not recommended, and you should rather use managed references.

Managed references

A managed reference is called a tracking reference and uses % instead of & .

void foo(int %i) {
  i = 42;
}

foo(f->result);

void foo(int %i) {

i = 42;

}

foo(f->result);

Managed pointers and references have some weirdness, for which this SO post goes into further details.

C++/CLI wrapping a C library: cheatsheet

Because once in a while I write a C++/CLI wrapper around a C library, but since I do it so infrequently I forget everything each time. Here is a list of pointers (haha) for common pitfalls and problems.

Compiling and debugging

You have to wrap the .h into an #ifdef __cplusplus extern “C” :

#ifdef __cplusplus
extern "C" {
#endif

// contents of the .h file

#ifdef __cplusplus
}
#endif

#ifdef __cplusplus

extern "C" {

#endif

// contents of the .h file

#ifdef __cplusplus

}

#endif

You also need to remove the /clr option for C files.
Right click your C files > Properties > C/C++ > General > Common Language RunTime Support > No CLR Support.

In your managed class, to include an unmanaged C header, use managed(push|pop)

#pragma managed(push, off)
#include "../../map_packer/packer.h"
#pragma managed(pop)

#pragma managed(push, off)

#include "../../map_packer/packer.h"

#pragma managed(pop)

Because Linux and Windows get along so well, you may also have to redefine a few C methods in your header file.

// Only for Visual C++
#ifdef _MSC_VER

// The Visual C++ compiler doesn't know "inline" but knows "__inline".
// see: stackoverflow.com/a/24435157/2354542
#define inline __inline

// Visual C++ is not POSIX compliant :'( so I have to code a new random() method
// rand() is not safe for security ; see: https://www.securecoding.cert.org/confluence/display/seccode/MSC30-C.+Do+not+use+the+rand%28%29+function+for+generating+pseudorandom+numbers
// but there is no security concern in my library, so I don't care much
inline long int random() {
    return rand();
}
#endif

// Only for Visual C++

#ifdef _MSC_VER

// The Visual C++ compiler doesn't know "inline" but knows "__inline".

// see: stackoverflow.com/a/24435157/2354542

#define inline __inline

// Visual C++ is not POSIX compliant :'( so I have to code a new random() method

// rand() is not safe for security ; see: https://www.securecoding.cert.org/confluence/display/seccode/MSC30-C.+Do+not+use+the+rand%28%29+function+for+generating+pseudorandom+numbers

// but there is no security concern in my library, so I don't care much

inline long int random() {

return rand();

}

#endif

If you wish to create a C# unit test project (using xUnit for instance), with Visual Studio 2012 and later, you will have to do a few things:

In the C++/CLI project’s properties, in Debugging, choose the Debugger Type “Mixed”
In the Visual Studio options, Debugging, check “Managed C++ Compatibility Mode” (in VS2012) / “Use Managed Compatibility Mode” (in VS2013).

To build using MSBuild, you may have to change the toolset to VS2012: project properties > General > Platform Toolset = Visual Studio 2012 (v110). And set the Configuration Type to Dynamic Library (.dll) while you’re at it.

Managed and unmanaged objects

gcnew instantiates a managed objects, which will be garbage collected. Always use it for the managed objects.

To easily call the native structures through managed classes, use the internal constructor and ToNative methods.

Powershell for data mining Access databases into SQL Server

I need to do some data extraction from several hundreds of Access databases extracted from zip files. The extracted data will allow us to do some statistics on our customer’s behavior.
In order to do that, I decided to use Powershell, because it has all the features I need, bundled in one neat language.

The scaffolding for the scripts uses PSake for the task launcher, Pester for Powershell unit tests, and a few Nuget packages (including PSake).

In a vendor folder, add the Nuget exe, as well as a packages.config file listing the necessary Nuget packages. I have added PSake, and NUnitOrange to transform the Pester results to a nice HTML report.
I’m not using the Pester Nuget package because it includes the Pester unit tests, and some of them fail, which breaks my build (in addition to adding a thousand tests I don’t care about). Instead, I directly include the Pester scripts, and I cleaned up the samples and tests.

Since the script needs to read Access databases using some sort of ADODB or OLEDB provider, you will have to install either the Access 2007 or Access 2010 provider. If you’re like me (with Office x86 already installed), you won’t be able to install the Access 2010 x64 provider, which will trigger “The Jet/ACE OLEDB provider is not registered on the local machine” errors. So you will have to run the x86 version of Powershell. Which means you will have to use the x86 version of the SQLPS module (make sure you download the x86 version). Don’t worry, the SQL 2012 version of SQLPS is compatible with older SQL Servers (at least 2008 R2).

The batch bootstrapper is inspired by the bootstrapper from Pester:

@echo off

rem Install nuget packages
%~dp0vendor\nuget.exe Install %~dp0vendor\packages.config -o %~dp0packages

SET INPUT=%~dp0input
SET OUTPUT=%~dp0output
IF NOT '%1'=='' SET INPUT=%1
IF NOT '%2'=='' SET OUTPUT=%2

rem Detect 32 or 64 bits Windows: http://stackoverflow.com/a/24590583/6776
SET PSPATH=powershell
reg Query "HKLM\Hardware\Description\System\CentralProcessor\0" | find /i "x86" > NUL && set OS=32BIT || set OS=64BIT
if %OS%==64BIT SET PSPATH=%SystemRoot%\syswow64\WindowsPowerShell\v1.0\powershell.exe

rem See: https://github.com/pester/Pester/blob/master/Build.bat
%PSPATH% -NoProfile -ExecutionPolicy Bypass -Command ^
    "$psakeDir = ([array](dir %~dp0packages\psake.*))[-1] ; Import-Module $psakeDir\tools\psake.psm1 ; Invoke-psake %~dp0tasks.ps1 Mining -properties @{'in'='%INPUT%';'out'='%OUTPUT%'}"

@echo off

rem Install nuget packages

%~dp0vendor\nuget.exe Install %~dp0vendor\packages.config -o %~dp0packages

SET INPUT=%~dp0input

SET OUTPUT=%~dp0output

IF NOT '%1'=='' SET INPUT=%1

IF NOT '%2'=='' SET OUTPUT=%2

rem Detect 32 or 64 bits Windows: http://stackoverflow.com/a/24590583/6776

SET PSPATH=powershell

reg Query "HKLM\Hardware\Description\System\CentralProcessor\0" | find /i "x86" > NUL && set OS=32BIT || set OS=64BIT

if %OS%==64BIT SET PSPATH=%SystemRoot%\syswow64\WindowsPowerShell\v1.0\powershell.exe

rem See: https://github.com/pester/Pester/blob/master/Build.bat

%PSPATH% -NoProfile -ExecutionPolicy Bypass -Command ^

"$psakeDir = ([array](dir %~dp0packages\psake.*))[-1] ; Import-Module $psakeDir\tools\psake.psm1 ; Invoke-psake %~dp0tasks.ps1 Mining -properties @{'in'='%INPUT%';'out'='%OUTPUT%'}"

The tasks in tasks.ps1 are pretty standard and minimalist, so that the maximum is tested in modules via Pester.

Reading Access databases is as simple as using OleDb, old-school style:

function Read-AccessDataSet (
  [string] $file,
  [string] $query
)
{
    $conn = New-Object System.Data.OleDb.OleDbConnection "Provider=Microsoft.Jet.OLEDB.4.0; Data Source=$file"
    $conn.Open()
    $cmd  = New-Object System.Data.OleDb.OleDbCommand($query, $conn)

    $adapter = New-Object System.Data.OleDb.OleDbDataAdapter $cmd
    $dataset = New-Object System.Data.DataSet
    $adapter.Fill($dataset) | Out-Null # returns the number of rows

    $conn.Close()

    return $dataset
}

function Read-AccessDataSet (

[string] $file,

[string] $query

)

{

$conn = New-Object System.Data.OleDb.OleDbConnection "Provider=Microsoft.Jet.OLEDB.4.0; Data Source=$file"

$conn.Open()

$cmd = New-Object System.Data.OleDb.OleDbCommand($query, $conn)

$adapter = New-Object System.Data.OleDb.OleDbDataAdapter $cmd

$dataset = New-Object System.Data.DataSet

$adapter.Fill($dataset) | Out-Null # returns the number of rows

$conn.Close()

return $dataset

}

And the corresponding Pester tests, using actual Access test databases:

$here = Split-Path -Parent $MyInvocation.MyCommand.Path

Import-Module "$here\db.psm1" -Force

Describe "database module" {
    $in = Resolve-Path "$here\..\tests\input"

    Context "reading Access DB" {
        It "should read multiple Access rows" {
            (Read-AccessDataSet -file "$in\input.dat" -query "SELECT * FROM Users").Tables[0].Rows.Count | Should Be 10
        }
    }
}

$here = Split-Path -Parent $MyInvocation.MyCommand.Path

Import-Module "$here\db.psm1" -Force

Describe "database module" {

$in = Resolve-Path "$here\..\tests\input"

Context "reading Access DB" {

It "should read multiple Access rows" {

(Read-AccessDataSet -file "$in\input.dat" -query "SELECT * FROM Users").Tables[0].Rows.Count | Should Be 10

}

Executing SQL scripts uses the much more powerful Invoke-SqlCmd cmdlet from the SQLPS module. The most is that it returns a Powershell object, so I can do something like this:

$users = Invoke-SqlCommand -query "SELECT * FROM users"

foreach ($user in $users) {
  write "id: $($user.id) ; name: $($user.name)"
}

$users = Invoke-SqlCommand -query "SELECT * FROM users"

foreach ($user in $users) {

write "id: $($user.id) ; name: $($user.name)"

}

Month: March 2015

C, C++ and C++/CLI cheat sheet

How to read pointers

In variable declaration

In variable usage

C pointers

C arrays

C structures

C++ references

Managed instances

Managed pointers

Managed references

Further readings

C++/CLI wrapping a C library: cheatsheet

Compiling and debugging

Managed and unmanaged objects

Powershell for data mining Access databases into SQL Server