DupeMan - Duplicate File Management Program Updated

It's been about 3 years since I've updated and released changes to my original DupeMan Duplicate file management tool, but I've been able to make changes to it in my spare time and when I get bored, I upgraded to .NET 2.0 framework and added the ability to save and load the results of a check for duplicates indexing process as well as a new MP3 Playlist dockable control.

Download DupeMan Here.



Feel free to pass this program or link on to your friends or anyone you think might be able to use it (those who's collection is more quantity than quality).

If you do install it and try it out, let me know any feedback you have,
-Matt Palmerlee

(Note: you need to have at least .NET 2.0 installed on your system to work.)

Configuring Google Apps to host your Domain for free

I've recently setup a new domain name for a new S/W Development Consulting company I'm creating called Mastered Software (MasteredSoftware.com). I was planning on creating my new website using C# .NET and Visual Studio .NET 2005, because that is the platform I'm most comfortable with, however, finding affordable Windows hosting that supports the .NET framework is not easy, the cheapest I could find was $10 a month if you bought 2 years at a time, which isn't much compared to other hosting solutions, but for a small business even this adds up quickly.

I was also recently checking out Google's App Engine and Google Apps for your domain services, and what's cool is that you can host your domain and even email through the google apps for your domain service! The best part is, for small companies that don't need lots of storage and features it's free!

Here are the steps I took to have my domain hosted by google:

1. Start by signing up for Google Apps for your Domain at: http://www.google.com/a/
2. You will be prompted to verify ownership of your domain during setup, there are a couple of options to do the verification, including adding a special CNAME record in your domain name registrar's system to point to google.com, this is the method I used for verification because it was the most straightforward. While you are there, you may want to add additional CNAME records for your domain which point to ghs.google.com (i.e. masteredsoftware.com and www.masteredsoftware.com).
   
3. Enable the applications you want such as email, calendar, etc... For creating the content for my main site I choose to use the Google Pages application because it's very easy, configure the web address for google pages to point to www. for my site it is www.masteredsoftware.com click on the "Change URL" link from the Web Page settings screen to update the web address. Create a basic home page for your site through google pages so you have something to look at when you go to your domain, don't forget to publish your changes so they are live on the site.
   
4. At this point you should be able to browse to the "www" version of your site (if you setup the CNAME records in step2 for www.yourdomain.com to point to ghs.google.com).
5. Unfortunately there is no way at the moment to have the "naked" domain version of your site point to your google pages site ,unless perhaps you have your domain hosted through google domain hosting, I don't know because I haven't tried that service. You maybe able to setup a forward or redirection from the naked domain to the www version in your hosting provider's DNS settings, with my hosting provider I was not able to do that and have some more of the advanced features for my domain such as redirecting the MX records to google. Fortunately you can host your naked domain using the google apps framework.
6. Once you have signed up for the google apps framework, add your naked domain (in my case masteredsoftware.com) under the google apps framework options.
7. Follow the instructions on this page: Uploading your application to upload your files to the google app engine service.
8. Here is the code for my masteredsoftware.py file:
   print "Status: 302 Moved"
   print "Location: http://www.masteredsoftware.com/"
   print ""
9. Also setup your app.yaml file pointing to the main script file:
   application: masteredsoftware
   version: 1
   runtime: python
   api_version: 1
   
   handlers:
   - url: /.*
   script: masteredsoftware.py
10. If everything worked out the code files above should forward your naked domain requests to your www.yourdomain.com version.
11. ...Finally you might want to add google analytics to your pages site so that you can track visitors. Just click on the area you want your analytics code, click the edit html link (on the bottom right of the google pages page), surrount the analytics code with div tags and save and publish your changes.
12. Now you can play with some of the other cool features that Google apps for your domain allows (such as gmail hosting, etc...)
13. Have Fun domain hosting!
    

VSS to SVN Migration

The last couple of weeks at work I've been moving our Source Control tool from Visual Source Safe (VSS) to Subversion (SVN). There are many reasons to move away from VSS, mainly it just sucks, random corruption, that nice error message about having to run the analyze tool, and just the fact that it is a very poor Microsoft Product. Also VSS doesn't support some of the features we want as we grow such as branching and merging.

There were other Source Control options we investigated when looking to migrate off of VSS, the most logical was to move to Microsoft's Team Foundation Server (TFS), which I've heard is very robust and is an easy switch from VSS, it also apparently has some really nice features for controling Database Schema and Code (Stored Procedure) changes. The problem with TFS was, to work effectively (in addition to Costly TFS licensing) our Visual Studio Developers would have to upgrade to the Visual Studio Team System which is an additional cost. Our decision was to go open source, SVN is a very popular technology for Source Control and it supports many of our desired processes for Source Change Management, such as branching and merging releases.


Here are some helpful links for SVN and the conversion from VSS:
Good Overview of the Upgrade Process

Subversion Main Site

Tortoise SVN Client

VSS to SVN Migration Tool

TamTam SVN Visual Studio SCC Provider Plugin

Since we wanted to retain the check-in history from Visual Source Safe, we used the VSS to SVN Migration tool listed above. I made some modifications to the tool as well, I wanted to keep the user who originally made the change as well as the date the change was made (the 2nd version of the Migration tool just uses one sign-on to SVN and just uses the current computer time for the check-in time).

My change parses the user name out of the Version information from VSS and uses this login instead for SVN, it also parses the date and time the checkin was made, switches the computers clock to that date and time, and then finally does the commit to the SVN repository (which is on my local machine anyways because I couldn't get the the program to work on Windows 2003). I did run into some problems where on some files I couldn't get the versions from VSS for some reason, maybe these files were from a backup/restore in VSS and that is why it had trouble, I'm not sure but I was able to manually import the files that had this problem and the rest of the process completed smoothly.

Here is the direct link to my Modified Version of PowerAdmin's VSS to SVN Migration tool (I sent it to them, but in case they don't repost my version, here it is):
VSSMigrate3_code.zip

The next step was to move my local SVN repository to the final destination server, which can easily be done with the Subversion dump and load commands as described on this page:
How to move a Subversion Repository


Finally, I've setup backup scripts on the Subversion Windows 2003 Server, we had a similar set of backup batch file scripts running to backup our VSS database and FTP the backup file offsite. These batch files were then setup to be run nightly and weekly via the windows scheduler. The new Subversion backup batch files perform a svnadmin hotcopy to make a copy of the repository without taking it offline, then it zip compresses the files from the hotcopy using the 7zip program's command line arguments, it finally creates a remote ftp script that is executed to ftp the compressed backupfiles offsite. Here are the contents of my fancy batch file backup script for Subversion:



FOR /F "tokens=1 delims= " %%i IN ('date /t') DO SET DATE=%%i

cd C:\Program Files\Subversion\bin\

RMDIR /S /Q D:\backup\SVN\current\
MKDIR D:\backup\SVN\current\

DEL D:\backup\SVN\%DATE%_SVN_backup.zip

svnadmin hotcopy D:\svn_repo D:\backup\SVN\current --clean-logs

"C:\Program Files\7-zip\7z" a -tzip -r -y d:\backup\SVN\%DATE%_SVN_backup.zip d:\backup\SVN\current

cd D:\backup\SVN\

> script.ftp ECHO username
>>script.ftp ECHO password
>>script.ftp ECHO cd ~/backup/SVN
>>script.ftp ECHO binary
>>script.ftp ECHO prompt n
>>script.ftp ECHO mput %DATE%_SVN_backup.zip
>>script.ftp ECHO quit
:: Use the temporary script for unattended FTP
FTP -v -s:script.ftp ftp.servername.com




After that developers must rebind their Solutions to SVN, which can be quite a pain, but once it's setup using TamTam, going forward it is smooth opening the solution files.
One tool that helped me out upgrade the solutions (especially VS2003) was Clean Sources Plus which removes source control bindings from the windows explorer right click menu.

ASP.NET Web Application Project Upgrade - Assembly Reference not found

We've recently undergone a conversion to Visual Studio 2005 Service Pack 1 (SP1) for the new Web Application Projects (WAP) they have added.

The Web Application Projects give more control over the files included in a Web Site because now you have a project file similar to the VS 2003 Web Application model.

There are a couple of nice guides online to upgrading a Web Site to a Web Application Project after installing SP1, here is one I followed for the Upgrade:
Scott Gu's Migrating a Web Site to Web Application Project.

Basically, you just create a new web application project, copy in all your pages and other files, then on all the ASPX pages, right click and choose "Convert to Web Application", which creates the filename.designer.cs files necessary for the new Web Application Model.

To simplify the upgrade initially I didn't add namespaces for the separate App_Code files and I just put them in a folder named "Classes".

After that there weren't many more hoops to jump through for the site to successfully compile.

One reason for the upgrade is to make it easier to create a Web Setup (Installer) for the Web Application Projects where making a Setup Project for the original "Web Site" project was a pain.

After creating the Web Setup Project for our Web Application Project, then testing the installer, I kept getting an error:
"The name 'Namespace' does not exist in the current context"
When I viewed the detailed compiler output: "The type or namespace name 'Namespace' could not be found (are you missing a using directive or an assembly reference?)". This was because the dll files for the website were not in the bin folder underneath the web application root folder.

Once I fixed this in the installer (I manually added the bin folder available from the "View -> File System" context menu, and added outputs: Primary Output for each web project and library as necessary).

Overall, I'm much happier with the new (or old) Web Application Projects model in Visual Studio 2005, it gives the developer much more control over the behavior of the website, the included files, simplifies deployment, and is much faster to compile and test.

Read Foreign Region DVD

I recently got back from a 3 week vacation to Spain and Italy, while I was over there we ran out of space on our camera xD drives so we went to an Internet Cafe in Italy and had them burn a DVD of our pictures that were on our 2 GB xD card. After that I nervously formatted that card so that we could continue taking pictures of our trip.

Once I got back to the States and started compiling all our photos (over 2000 of them!) I insert my DVD with my photos into my computer's DVD drive and I hear it clicking and thrashing and my heart drops into my stomach.

When my DVD ROM couldn't read the disk I paniked, I tried to keep my calm; telling myself it was alright, the files were there, I just had to get them off it.

After much searching online for a solution, I realized the problem was my DVD-ROM was the 2nd generation computer DVD Player with this Movie Industry Region protection so the movie industry can control releases of movies within different countries. What a crock!!!

Luckily I was able to find a firmware upgrade tool that allows my model of DVD Rom to be essentially Region-Less (or region 0) so the DVD player doesn't care what region the DVD is actually encoded for.

After I upgraded the Firmware and used this DVD Region Killer s/w to remove the region protection from windows I could finally read my dvd and use windows to get my photos off of it.

Note: do not leave a foreign region DVD in the drive on bootup or you are bound to get hangs as the O/S looks at the drive to boot from.

HttpWebRequest in C# for Web Traffic Simulation, Watch out for Expect 100 Continue and Nagle

Over the past few months we've been busy at work creating our scalable Intelligent Chat application using C# .NET 2.0 and AJAX. To perform load testing against our application we have written a simple Traffic Generating/Load Simulating Windows Forms application which makes Asynchronous HTTP Requests using the HttpWebRequest object, it is pretty straightforward and has been able to surface many problems in our main application. We did come across a big "Gotcha" in our simulator, after looking at the generated network traffic we noticed that in the header of each request the simulator was sending "Expect: 100 - Continue", after which the server would send the 100 - Continue response to the client. Since this was not the correct behavior, we added the following line of code:
System.Net.ServicePointManager.Expect100Continue = false;
As indicated in the following blog posts:
HttpWebRequest and the Expect: 100-continue Header Problem
HttpWebRequest and Expect 100 Continue

After this change however, the headaches started, everything was fine when we tested in our Development environment, IIS on Windows 2003, when we deployed the simulator into the Test environment (as far as I can tell, exactly the same server configuration, etc...), the simulator was so slow that requests would time out almost immediately.

After using Wireshark to view the network traffic in the test environment, we were seeing some strange behavior, it looked like the simulator was sending the header of the POST request, and then waiting for the Acknowledge (ACK) message back (which would take more than 200 milliseconds) from the server before sending the body of the post. Thanks to our brilliant network engineer we were able to determine that the client caused this behavior by using the Nagle Algorithm for it's requests.

That is when we put in one more line:

System.Net.ServicePointManager.UseNagleAlgorithm = false;
In our simulator and it seemed to fix the problem in the test environment, I have no Idea what is different in our development environment, some IIS configuration, maybe the fact that our test environment servers have .NET 3.0 installed? It is still a mystery to me. I figured I would share our experiences with the Expect 100 continue and Nagle Algorithm to perhaps save others some of the same pain we dealt with debugging this issue.

Update September 7th, 2007:
We found the difference in our development environment, the windows 2003 server in our development environment is R2 with Service Pack 1 (sp1) and the windows 2003 server in the test lab is win 2003 R2 with Service Pack 2 (sp2), after viewing further network traces this seems to stem from IIS sending a 100 continue under sp1 (even when we don't send the expect 100 continue message), then what we see is the client waits a long amount of time before sending the ACK message back to the server (I suspect because it has already sent the entire body in the initial POST headers).

This Blog post points to a hotfix that addresses this issue in sp1:
HTTP.SYS, IIS, and the 100 continue

I've put together a simple table to show all the combinations related to 100 - Continue and Nagle, and the results of the test (it was just too hard to keep it all straight in my head):

Results	Server Sends 100 - Continue (Windows 2003 SP1 && XP IIS 5.1)	Client sends Expect: 100 - Continue	Use Nagle Algorythm
Fast (extra traffic sent)	1	1	1
Fast (extra traffic sent)	1	1	0
Fast (didn't send POST body, after the 100 - Contine it has something to send?)	1	0	1
Slow (waits before sending ACK to 100 - Continue)	1	0	0
Fast Responses	0	1	1
Fast (extra traffic sent)	0	1	0
Slow (Nagle causes a ~200ms wait for ACK)	0	0	1
Fast Responses	0	0	0

(Note: A zero in column one means Windows 2003 SP2)

SQL Server Outer Join with where clause bug

I spent most of the day trying to figure out one simple outer join query in SQLServer 2005, it turns out that the problem stems from the WHERE clause, apparently SQL Server will execute the WHERE clause before the join? Which doesn't make much sense to me, it seems like a bug, but whatever, at least I only wasted about six hours trying to figure out that once you put the criteria within the join clause it works as expected.

Here are some more articles I found on the subject:
Outer Join trouble
Additional Criteria in the JOIN Clause

C# DrawImage and overriding the OnPaint method

I recently spent hours looking for a solution to a strange problem I had in my custom user-drawn control when using the DrawImage method, and I thought I'd share my misery and hopefully keep you from wasting your time figuring this out as well. What was happening was that using images that were not 96 dpi were scaled wrong using the DrawImage method in C# (I also tried using the DrawImageUnscaled methods to no avail). I figured that the DrawImage method should account for the different screen resolutions and I wouln't have too, but it turns out that there is a SetResolution method on the Image (Bitmap object). So I delved further on how to get the system's resolution (or dots per inch, dpi) from .NET and I ran across this page:
How to: Handle Orientation and Resolution Changes that talks about the Microsoft .NET Compact framework, but there was a very interesting tidbit near the bottom of the page:

"If your application contains graphics that are drawn in the OnPaint method, they will not scale automatically. You will need to use the DpiX and DpiY properties of your Graphics objects to determine appropriate scaling."

That was all I needed, one line in my OnPaint method and my images are now scaled properly:
(g is my Graphics object)

this.myBitmap.SetResolution(g.DpiX, g.DpiY);


It's amazing how long it takes to find the answer to questions when you don't know how to ask google for them!