Squashing range of old commits in git

    Changing the commit history of your branch is usually not the best idea. It should be done only in very specific cases. First of all by changing the history you will cause a great headache for whoever has changes in a branch branched off of the branch you are changing. My case was very specific. I had to squash old commits in the develop branch, which was used earlier by pretty much just a single developer. The effort was to create a demo application, which prompted the thought to just hide all these commits behind one "demo preparation" commit. Here are the steps I've taken to perform the squash on selected commits of the branch. I had a freshly installed git on a windows machine at my disposal.


1.      I've branched off of develop and created develop-squash, a branch I could work on
 
2.      I've cloned the same repository in a separate location - I like to have an option to manually check different commits in the other repository, while I'm working on a rebase in my initial repo
3.      I've set the default git text editor to Notepad++:
 
git config --global core.editor "'C:/Program Files (x86)/Notepad++/notepad++.exe' -multiInst -notabbar -nosession -noPlugin"


4.      I've launched interactive rebase to change the whole history of the branch:


git rebase -i --root


What it will do is it will open the interactive rebase file in Notepad++. We will get a list of all commits in the branch. --root makes it list all commits from the very first one. We can set a specific hash instead of root, if we want to change the commit history from specific commit and not the first one. The list will not contain merge commits from other branches.


Each commit in the list is set to "pick" which means the commit will be picked in the rebase operation. If we want to squash the given commit we need to change to "squash" or "s" for shortcut. (we can also drop specific commits or reorder them as we see fit).


5.      After we've finished preparing the rebase, we can close the notepad document. Git will see it and execute the rebase, which will start going one by one. Each time there's an issue git will stop the rebase and allow us to act.
 
6.      Whenever the rebase is stopped we get a chance at seeing what exactly went wrong and solve it. What usually happens is git tries to flatten the history behind the sub-branches of the branch we're squashing. In this case it will require you to resolve all conflicts. After you've resolved the conflicts with one of many diff tools, you can continue the rebase.


7.      Continue the rebase by executing:


git rebase --continue


Or abort it:


git rebase --abort


8.      In my case quite often, to get sure I'm merging correctly I was checking a later point in the commit history, in the second repository I've cloned earlier.


9.      After the rebase finished I've switched the names of the branches, created develop-legacy out of the initial develop, and  develop branch out of the develop-squash branch like this:


git branch -m develop develop-legacy
git branch -m develop-squash develop


10.   Sort out master branch and HEAD:


I've deleted the remote develop branch and pushed the new one in. Then I've overwritten the remote master branch with my develop as well:

git push -f origin develop:master

The above will also set the HEAD index onto this new master, which will leave our branch structure how it should be.


11.   Cleanup


At this moment we still heave our develop-legacy branch, which is the old branch with all our old commits. If we remove it we can pretty much remove all the associated tags and sub branches. The reason being - we won't need this history anymore, since we've already rewritten it.


If we'll only delete the develop-legacy branch - our old commits will still be hanging, being kept by either old tags or sub branches. Once there are no branches and tags associated with it, git garbage collector should remove the orphaned commits (we can also force run it as well).

All in all, there are most probably better ways of doing it. This was something I've cobbled together. I'd be happy to learn about other options which I'm sure git has in stock.

Adding tiled watermark text to documents

        Recently I really wanted to start adding watermark texts to different documents with my personal information in them. These are usually related to the hiring process. There are different documents, which we frequently send out whenever we are getting x-rayed by this or that company, which is trying to hire us.

A watermark can be helpful in ensuring that the documents we send, if they ever were to appear somewhere, somehow, are going to be easily traceable back to the source. Obviously it is possible to remove a watermark especially with a shabby quality photo-copy. But at least it adds this 1 extra step which perhaps some people will not be willing to go through.

A watermark text itself could contain the information to whom this document is being sent to, and who has created it.

I wanted to be able to scan any document I want on my home scanner, have the document in a JPG format and add a sliding, tiled watermark text of my choosing. I also wanted to be able to script this operation for multiple files.

Obstacles:

- all printers I had access to printed in PDF

- there's no free, simple way of converting PDF to JPG without sending your data to some sketchy website

- adding watermark automatically to multiple documents (otherwise the easiest way would be to use a free tool such as GIMP to add such watermark text manually)

I've done a bit of digging and the simplest solution for me was as follows and requires installation of 2 programs:

- ImageMagicks (a console based image processing app)

- GhostScript (a pdf renderer)

ImageMagicks has it's own pdf to jpg conversion command but it relies on having GhostScript installed underneath - you can use GhostScript solely for that for better performance for batch jobs. I've used the ImageMagicks version for ease of use.

Once all is installed you can use the following command to convert pdf to jpg, given there's a Watermarking directory in your ImageMagics folder:

convert 
-density 150 
-trim Watermarking\test.pdf 
-quality 100 
-flatten 
-sharpen 0x1.0 Watermarking\test.jpg

You can customize the quality using different parameter values, for more info check: https://www.imagemagick.org/script/convert.php

Next we can add a watermark to the resulting image:

convert 
-font Arial 
-pointsize 40 
-size 430x270 xc:none 
-fill #80808080 
-gravity NorthWest 
-draw "rotate 15 text 5,0 'Mr XYZ Company ABC'" miff:- | composite 
-tile - Watermarking\test.jpg Watermarking\watermarked_test.jpg

You can read more oh the parameters used above under the following url: http://www.imagemagick.org/Usage/annotating/#wmark_text

Example pre-watermark:




Example post-watermark:


With the parameters in the command line you can among others set the density of the text it's size, size of the canvas the text is first created in, pivot the text and set the color of the font.

Once you have these commands it's really easy to parameterise them and use powershell, batch or other scripting tool to run it for all pdf's/jpg's in the given folder.

Troubleshooting:
- When on Windows you might want to pass the full path to the Convert.exe, otherwise Windows may mistake the call for another convert.exe which is a system executable related to the filesystem.

Minimal unit test setup Visual Studio 2017, Karma, Jasmine, Angular2, Typescript

Coming from the .net world setting up a first unit test for my angular side project was a very painful experience. Due to the sheer amount of frameworks and configuration I had to put a lot of time into it and spend a lot of time googling. The number of frameworks doesn't help and it's difficult to find exactly the same combination that you've chosen for your project. I've decided to use Karma because it was used in one of the first Visual Studio + Karma + Typescript tutorial I've found (later I've learnt from my close friend that it wasn't the best of choices).

I've decided to use the Visual Studio 2017 + ReSharper, first because I love these tools, second because I had my Azure WebJob and my WebService projects in the same solution as my angular2 based website.

It took a lot of time for me to go through it (after quite a bit of time I was able to run my tests from ReSharper but my goal was to make it run using either gulp tasks or NPM scripts tasks. After many retrials and tests I've narrowed down my Karma config (karma.conf.js) to the one below:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

module.exports = function (config) { config.set({ frameworks: ['jasmine', 'karma-typescript'], files: [ 'tests/**/*.js', 'tests/**/*.ts' ], karmaTypescriptConfig: { compilerOptions: { module: 'commonjs', target: 'es2015' }, tsconfig: 'tsconfig.json' }, preprocessors: { '**/*.ts': ['karma-typescript'] }, reporters: ['progress', 'htmlDetailed'], plugins: [ 'karma-phantomjs-launcher', 'karma-jasmine', 'karma-typescript', 'karma-html-detailed-reporter' ], htmlDetailed: { splitResults: false }, port: 9876, colors: true, logLevel: config.LOG_INFO, autoWatch: true, browsers: ['PhantomJS'] }); }

Most of the errors I was receiving stemmed from the miss-configuration, and the fact that I was missing some dependencies. Karma is very configurable and most probably using something more simple like Tape would have set me off much faster.

Others steps I've taken to make it run/troubleshoot:
- installed Task Runner Explorer extension, which gives a nice gui for the tasks
- I've checked running the tests both in PhantomJS and headless Chrome
- when I was looking for errors I've switched logging level to Verbose, being able to see the logs being displayed in the Task's Runner output was something, which has greatly helped me pinpoint different issues
- using karma-typescript pre-processor for the ts files along with its config helped me with a lot of errors caused by unexpected token errors
- many of the errors were related to the incorrect module system or ecma script version
- first unit test (or spec file) which I've successfully launched was also my 'proof of concept' and was really simple:

1 2 3

describe('1st tests', () => { it('true is true', () => expect(true).toBe(true)); });

Set up page reload/type script compilation on save in Visual Studio 2015 with typescript

      The goal is to be able to transfer the changes done in TS code during debugging onto the running website. This is possible in Visual Studio using the Browser Link functionality. Whenever we debug and we've got Browser Link Dashboard open - we should be able to see our browser of choice during the debug session listed under specific projects.

This can actually allow us to have the same project being run in multiple browsers and refreshed in all at the same time.

Straight away we learn about the particular items which must be fulfilled to make it work:

1.       Static HTML files linking should be enabled by adding the following to the Web.config:
 

<system.webServer>

<handlers>

<add name="Browser Link for HTML" path="*.html" verb="*" type="System.Web.StaticFileHandler, System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" resourceType="File" preCondition="integratedMode" />

</handlers>

2.       Debugging must be enabled in the Web.config file (this will typically be added to your web.config by default)

<system.web> <compilation debug="true" targetFramework="4.5.2"/>
 

3.       IIS server must run on .NET 4.0 or later.
 

4.       Compile on save has to be enabled in project options or particular setting in the csproj file has to be set (this can actually be changed mid-debugging):

Project properties:

Project file:

<TypeScriptCompileOnSaveEnabled>True</TypeScriptCompileOnSaveEnabled>

5.       Server side caching (files cached by IIS)

Add the following line to Web.config to instruct IIS not to cache files:

You need to set

<caching>
   <outputCache enableOutputCache="false" />
</caching>

or if its IIS 7+ (Which IIS Express will be)

<system.webServer>
    <caching enabled="false" />
</system.webServer>

6.       Client side caching (in-browser caching)

Chrome:
An option is to "Disable cache" in Dev Tools -> Networking
 

This will however force you to have Dev Tools open while debugging. There's a similar functionality for FireFox where caching can be only disabled for as long as long the developer tools are opened. To simplify working with such setup it's good to open Dev Tools in a separate window - even if you're not using it in this specific case you can keep it opened in the task bar.

7.       There are numerous examples of running Chrome in a non-caching mode, neither worked for me (chrome opened from command line with incognito mode, disabling application cache with an argument or outright setting the cache limit to a very small number. In my case it only worked for the first index file opened but the template html files used in components were not getting refreshed.

Environment used:
Visual Studio 2015 Update 3

Resharper 2016.2

Project created using Angular2WebTemplate

Someone's workaround:

http://stackoverflow.com/questions/41108995/using-angular-2-vs2015-and-iis-how-can-i-get-my-html-template-changes-to-refl

The "IsFileSystemCaseSensitive" parameter is not supported by the "FindConfigFiles" task error.

    I've recently started working on a small project with angular used on the front end. Part of it was adding the project from template (I've used a popular scaffolding Angular2WebTemplate). I've installed required nugets: Microsoft.TypeScript.Compiler and Microsoft.TypeScript.MSBuild.

After first build I've received the following error:
The "IsFileSystemCaseSensitive" parameter is not supported by the "FindConfigFiles" task error.

It took me a bit of digging to find this:
https://github.com/Microsoft/TypeScript/issues/15536

The issue seems to have been caused by two imports covering 2 different version of the same targets file. The solution was to remove one of the imports. Below a solution pasted from the above url:

Remove the local import (or Nuget import which ever one you choose). 

• local

 <Import Project="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\TypeScript\Microsoft.TypeScript.targets" Condition="Exists('$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\TypeScript\Microsoft.TypeScript.targets')" />

• Nuget - should be near the end of the proj file

 <Import Project="..\packages\Microsoft.TypeScript.MSBuild.2.3.1\build\Microsoft.TypeScript.MSBuild.targets" Condition="Exists('..\packages\Microsoft.TypeScript.MSBuild.2.3.1\build\Microsoft.TypeScript.MSBuild.targets')" />

In my case I've removed the first entry which was enough to make the project compile and run.

Object copying speed comparison, Part 2

      As planned - I'm adding another part of the overview of different methods used in the previous post. I'll try to dive into different nooks and crannies and see what I can learn from it. Before that I'd like to address questions raised.

1. It is unfair to compare converting done in parallel and the one done solely with AutoMapper. And unfair it is indeed. My goal however was to have an interesting learning experience. I wasn't planning to have a race.

2. Some of the cases don't make sense. And again it's correct. Most of them don't. Most of the time it's not the code that does objects conversion, which is our production code's bottleneck. Most of the time we'll find our performance issues elsewhere. Similarly as above - we're trying to have fun here and learn something new.

Going back to the samples:

1. ManualMap - simple creation of the view models objects and copying of the data. First thing, which can increase the performance is setting the capacity of the output list to a predetermined size. Bear in mind this will become visible only with the increase of the items count.
   
                 
   
   The cause for this is memory reallocation and GC.
   Here's a good explanation: http://stackoverflow.com/questions/2247773/is-it-worthwhile-to-initialize-the-collection-size-of-a-listt-if-its-size-rea
2. ManualMapFor - uses a for loop to iterate through the collection. In our case it was ~30%
   faster than the foreach version. Depending on the underlying collection the results may very. Iterating through arrays should be fastest due to the generated IL optimizations.
   Here's a good read on specifics:
   http://codebetter.com/patricksmacchia/2008/11/19/an-easy-and-efficient-way-to-improve-net-code-performances/
3. AutoMapperMap - this case uses an AutoMapper library, which allows us to workaround the boilerplate code of copying the objects. It requires initial configuration - and it is slower, than using a manual copy adapter class. What we gain is the lack of all the extra boilerplate. Based on the results it was 10 times slower, than using manual adapter with a foreach loop.
   
   It would have been slightly faster, if we used a for loop over an array.
4. AutoMapperCollectionMap - the mapping is always faster when we register a collection within the mapper. Performing a mapping of collection of objects to another collection of objects. Whenever we use Automapper that's the recommended way of performing the mapping.
5. LinqMap - the most basic comparison to do was to compare it against an ordinary foreach loop. The LinqMap does a simple Select to do the mapping and ToList at the end. I expected it to be slower. This is how the comparison looks like:
   
   
   
   Select statement based on the collection it is running on, internally chooses an iterator, hence depending whether it's list or array we might get a speedup. Additionally Linq statements such as Select and Where use iterators, which allow for optimizing consecutive chained Where and Select calls. The logic behind them adds additional performance hit.
   
   To be continued...

General optimizations:

1. Many .NET library methods perform optimizations based on type of collections we pass as a parameter. When we use the generic IEnumerable many will perform a check to see whether the collection implements ICollection, IList or whether it is an array. Knowing the more specific interface allows to use methods which make faster execution possible. Reads and writes themselves can differ in speed due to how out of bounds checks are performed in different cases. In the test below a foreach loop has performed manual copying over an array of dtos and a list of dtos. The array version is visibly faster.
   
   
   

Object copying speed comparison

       I've been wondering about speed differences in execution of property copying code, which is usually used/found in the adapter pattern implementation. The answers are obviously in stackoverflow and in various blogs. But I thought about pushing it a bit further and comparing speeds of different approaches to this issue.

Bear in mind different ways of doing the same thing have advantages and disadvantages and each way has different caveats. I've selected a couple of approaches, there are many which I've either missed or disregarded. Some of the listed methods have minimal differences between them - or would never be used in production code.

As a common use case for all I've selected copying data between a data transfer object (DTO) object and a View Model object.

1. ManualMap - an ordinary property by property copying of whatever is in the object. Most often this is how it's done manually. Uses a foreach loop to loop through the objects.
2. ManualForArrayMap - takes in an array of Dtos and uses a for loop. It could use a list as well.
3. LinqMap - same as the foreach one but uses Linq to loop through the items.
4. AutoMapperMap - uses automapper and foreach loop to loop through all the items
5. AutoMapperLinqMap - similarly as above but uses Linq
6. AutoMapperCollectionMap - maps the whole collection in one go instead of looping through the items and mapping one by one.
7. ILMap - uses Emitted code which is cached as a delegate. Generic.
8. ExpressionMap - uses expressions to bind properties of the objects. Generic.
9. ReflectionOrderedPropertiesCopy - uses reflection with the assumption that the properties are ordered. Generic.
10. ReflectionPropertySearchCopy - uses reflection and searches for each property within the object by name. Generic.
11. ParallelForEachManualMap - uses a partitioner and a parallel foreach loop to split the collection into separate ranges.
12. ParallelLinqMap - parallel linq with the use of AsParallel method. Checked a couple of times with different degrees of parallelism set.

The slowest out of these were the ones which use reflection (9 and 10), I haven't added them in the comparison cause numbers were off the chart. 7 and 8 use reflection as well but only when building the lookup - the time required for it is not taken into consideration, similarly as AutoMapper configuration.

To check the timings I've used DotNetBenchmark library, I've run it in release outside of Visual Studio to have fairly correct results. All the results below are actually a median taken out of multiple runs of the same method.

Below you can see the comparison of timings for different numbers of DTOs fed into the methods, all of these timings are in nanoseconds.

Comparison of timings.

Next I've created a chart based on the above.

There are some obvious takeaways above. I'll go into more details in the next post.

I'm pretty sure I have missed something or introduced some errors in the code - if you notice anything feel free to let me know. The code itself is sometimes under optimized and sometimes over optimized - but it was not the point of this specific post - it was more getting the general feeling of how fast can this boilerplate code run.

Links:

Github with sources for the comparison project: https://github.com/simonkatanski/speedtest/

BenchmarkDotNet: https://github.com/PerfDotNet/BenchmarkDotNet