Wednesday, November 14, 2007

VSS to SVN Migration

The last couple of weeks at work I've been moving our Source Control tool from Visual Source Safe (VSS) to Subversion (SVN). There are many reasons to move away from VSS, mainly it just sucks, random corruption, that nice error message about having to run the analyze tool, and just the fact that it is a very poor Microsoft Product. Also VSS doesn't support some of the features we want as we grow such as branching and merging.

There were other Source Control options we investigated when looking to migrate off of VSS, the most logical was to move to Microsoft's Team Foundation Server (TFS), which I've heard is very robust and is an easy switch from VSS, it also apparently has some really nice features for controling Database Schema and Code (Stored Procedure) changes. The problem with TFS was, to work effectively (in addition to Costly TFS licensing) our Visual Studio Developers would have to upgrade to the Visual Studio Team System which is an additional cost. Our decision was to go open source, SVN is a very popular technology for Source Control and it supports many of our desired processes for Source Change Management, such as branching and merging releases.


Here are some helpful links for SVN and the conversion from VSS:
Good Overview of the Upgrade Process

Subversion Main Site

Tortoise SVN Client

VSS to SVN Migration Tool

TamTam SVN Visual Studio SCC Provider Plugin

Since we wanted to retain the check-in history from Visual Source Safe, we used the VSS to SVN Migration tool listed above. I made some modifications to the tool as well, I wanted to keep the user who originally made the change as well as the date the change was made (the 2nd version of the Migration tool just uses one sign-on to SVN and just uses the current computer time for the check-in time).

My change parses the user name out of the Version information from VSS and uses this login instead for SVN, it also parses the date and time the checkin was made, switches the computers clock to that date and time, and then finally does the commit to the SVN repository (which is on my local machine anyways because I couldn't get the the program to work on Windows 2003). I did run into some problems where on some files I couldn't get the versions from VSS for some reason, maybe these files were from a backup/restore in VSS and that is why it had trouble, I'm not sure but I was able to manually import the files that had this problem and the rest of the process completed smoothly.

Here is the direct link to my Modified Version of PowerAdmin's VSS to SVN Migration tool (I sent it to them, but in case they don't repost my version, here it is):
VSSMigrate3_code.zip

The next step was to move my local SVN repository to the final destination server, which can easily be done with the Subversion dump and load commands as described on this page:
How to move a Subversion Repository


Finally, I've setup backup scripts on the Subversion Windows 2003 Server, we had a similar set of backup batch file scripts running to backup our VSS database and FTP the backup file offsite. These batch files were then setup to be run nightly and weekly via the windows scheduler. The new Subversion backup batch files perform a svnadmin hotcopy to make a copy of the repository without taking it offline, then it zip compresses the files from the hotcopy using the 7zip program's command line arguments, it finally creates a remote ftp script that is executed to ftp the compressed backupfiles offsite. Here are the contents of my fancy batch file backup script for Subversion:



FOR /F "tokens=1 delims= " %%i IN ('date /t') DO SET DATE=%%i

cd C:\Program Files\Subversion\bin\

RMDIR /S /Q D:\backup\SVN\current\
MKDIR D:\backup\SVN\current\

DEL D:\backup\SVN\%DATE%_SVN_backup.zip

svnadmin hotcopy D:\svn_repo D:\backup\SVN\current --clean-logs

"C:\Program Files\7-zip\7z" a -tzip -r -y d:\backup\SVN\%DATE%_SVN_backup.zip d:\backup\SVN\current

cd D:\backup\SVN\

> script.ftp ECHO username
>>script.ftp ECHO password
>>script.ftp ECHO cd ~/backup/SVN
>>script.ftp ECHO binary
>>script.ftp ECHO prompt n
>>script.ftp ECHO mput %DATE%_SVN_backup.zip
>>script.ftp ECHO quit
:: Use the temporary script for unattended FTP
FTP -v -s:script.ftp ftp.servername.com




After that developers must rebind their Solutions to SVN, which can be quite a pain, but once it's setup using TamTam, going forward it is smooth opening the solution files.
One tool that helped me out upgrade the solutions (especially VS2003) was Clean Sources Plus which removes source control bindings from the windows explorer right click menu.

10 comments:

JP said...

Hi,

Thanks for your vss-svn migration tool. I have a couple questions while using your tool:
1) I've run into a problem where migrating certain projects owned by others doesn't quite work since access to SVN requires Windows Authentication, and I can't obviously move others' projects (using their username) into SVN since I don't have their pwd's. Do you know if there is a way to perhaps change SVN access (on the SVN server) to use a set pwd for everyone or just turn off authentication process altogether so that we can bypass this issue?
2) My understanding is that currently, your tool does not migrate over the labels to SVN. Is there a way for us to do that?

Thanks much,
Jay

Matt said...

Jay,

Since the idea of labels doesn't really exist in Subversion, the tool does not migrate those, in Subversion you have branches instead which are a bit different.

As far as your problem with the subversion users and authentication, you maybe able to work around that by creating a local svn repository with access for the users and then moving the repository to the production server later and then setup the SVN configuration for authentication after the import.

You may also want to check out the original SVN migrate tool here:
http://www.poweradmin.com/SOURCECODE/VSSMIGRATE.ASPX
I think there is only one account it uses to connect to svn (but then you loose the user data who originally checked it in.

Hope this helps,
-Matt

Rafael said...

Hi.

We are planning to move our project from VSS to SVN, but also there is another application in the middle that uses some information from the comments to properly open the files.
I already test the original migrtaion tools, and it seems id makes a good job migrating my project, but as I'n not a C programmer I couldn't understand it's code very well.
I see in your comments, that your changes uses the user and the data when the original commit was done, this is something we need too.
The comments I need to add are basically related with the changes you already have done.
I need to add this information in the comments for SVN:
SVN Revision:
USER:
vss label: (this information is not necesary, but useful if it exists)
Comments (this is already migrated)

Could you please guide me where can I implement this changes in your source code?

Thanks

mpalmerlee said...

Rafael,

You want the svn revision number and the user in the comments of the check-in to subversion? These are properties that are already separately stored by subversion, I'm not sure I understand why you'd want them in the comments also, but I'll try to help nonetheless.

Unfortunately I'm not much of a c++ programmer either, but I know enough to read the syntax and make minor changes, for the label changes you would probably want to look at line 423 in VSSMigrate.cpp (GetAndAddFiles function) where it calls the command line source safe exe, perhaps there is a command line argument you can pass to include labels (or perhaps getting labels is a separate command, I'm not sure). Basically you'll want to read the labels and the other data you want in the comments, and stick them into the msg variable (see line 495 for where it actually posts that comment to subversion).

Hope this helps,
-Matt

Ramesh said...

Hi,

Thanks for sharing your work. I'm running into a nagging problem though. The program migrates the directory structure fine, but it gives this error message on every revision of every file:

The system cannot accept the date entered.
Enter the new date: (mm-dd-yy)

Any ideas?

Thanks again.

Matt said...

I'm not sure about the date problem, are you running the upgrade according to the instructions in my post. I ran into problems running the upgrade directly on a windows 2003 machine I think.

JP said...

Hi Matt,

Re: you solution, "As far as your problem with the subversion users and authentication, you maybe able to work around that by creating a local svn repository with access for the users and then moving the repository to the production server later and then setup the SVN configuration for authentication after the import," even if I create a local svn repos, i will still need the project owners' Windows pwds to migrate the VSS data over to my repos. So, I'm not sure how your solution would work. Is there a way to turn off the SVN authentication during the migration so that your script will bring everyone's projects over w/o authenticating? I just need my end result to be having all of the VSS data w/ the owners' names and the correct time stamp on all the check-in history in SVN.

Thanks for your help.
Jay

JP said...

Hi Matt,

I'm still dealing with my issue #1 I posted a while ago. I was wondering, how did you define the httpd.conf, svnserve.conf, config.ini files to migrate VSS records to SVN? Did you have any issues with authentication issues for projects owned by different people? I'm really at a loss.

This is how I defined the httpd.conf file:

(Location /svn) (changed to () here so it will post)
DAV svn
SVNParentPath C:/SVNRepos

# authentication
AuthType SSPI
AuthName "Subversion Repositories"
SSPIAuth On
SSPIAuthoritative On
SSPIDomain myDomain
SSPIOfferBasic On
Require valid-user
(/Location) (changed to () here so it will post)

Perhaps, I can use a different AuthType that helps me bypass this issue, while keeping the history and Owner's names of the VSS records? I can easily bypass this but it does not transfer over the Owner's names.

I'd appreciate your help.

Thanks,
Jay

Matt said...

JP-
Make sure you've created all the users you have in vss within subversion. These users should have the password the corresponds to the password you've set for the SVNPASSWORD value in the config.ini file.

Also, I'm not using HTTP authentication in subversion for the migration process. Perhaps look into a different authtype for svn, I can't remember what I use but I think it's the default for the svn server.

hope this helps,
-Mat

Ramesh said...

It looks like the problem was with date format in my VSS database. Karl's version of VSSMigrate did finish the migration, but all the dates came out wrong. So I modified his code to recognize "MM-DD-YY" date format and it worked. Guess I could've made the same modifications to your code, but I was in a rush to get the migration out of the way. My changes are available at: http://rameshsabeti.blogspot.com/2008/10/vss-to-svn-migration.html