Tuesday, October 25, 2011

Migration Clearcase to Git -- part 2

Several days ago I had a post to record the unsuccessful experience to migrate source code from Clearcase to Git.

We have a new way after doing some brain storms. This way still is not a perfect solution, but it's much better than previous trial.
  1. Use clearexport_ccase to export the source folder to intermittent data. See documentation of Clearcase admin.
  2. Create a temporary vob for importing the data later. See example.
  3. Import the data into temporary vob. See example.
  4. Repeat step 1 to 3 for importing all necessary data into temporary vob.
  5. Use the SVN Importer to import the temporary vob as Subversion repository.
  6. Last steps refer to a documentation of succeeded migration case of one of Eclipse project from Subversion to Git.
Git definitely is greatest SCM tool now. The size of Subversion repository is around 10GB, finally the Git repository is less than 700MB, which saves more than 10 times disk space. It's awesome!

The flaw of this way is that the removed elements in Clearcase(said using Main/LATEST as cspec of Clearcase vob when exporting) would lose after importing into a temporary vob. So switching to a maintenance branch or tag like 1.0/2.0 in Git, the source code is incomplete. The files existed in that branch or tag, then removed in latest code base are lost. The workaround could be manually checking in GA version to have complete code.

If anybody have graceful and perfect solution to migrate Clearcase to Git, I think he could start a new business. :)

Monday, October 17, 2011

Migrate Clearcase to Git

I tried to migrate the source code of project from Clearcase to Git repository. As far as I know there is no elegant solution for such migration. For purpose of this migration, I want to keep the history and label of files in Clearcase after migrating to Git repository.

There are mature tools to migrate CVS/SVN repository to Git, so I tried to use Subversion as a bridge for my migration.

I used a free software 'SVN Importer' to import the Clearcase vobs to Subversion. The tool is great, and it keeps the history of files, labels and branches. The entire size of new Subversion repository has near 50GB which is unacceptable size of Git repository. The subversion repository contains a lot of legacy code and unwanted binaries, so removing those revisions could significantly reduce the size of subversion repository. And subversion provides some admin tools to manipulate the metadata of subversion, it's possible to remove the unnecessary revisions and re-create a subversion repository with refined content. But I don't have any experience to use the admin tool of subversion before, I failed to filter the unwanted data. It's not worthy of costing too much effects on it. Finally I failed to filter the subversion repository.

Actually the detail history of files is rarely used. If need, we still can find it in Clearcase. At last I manually checked in the released version of our project into Git repository, and tagged them.

Wrote this unsuccessful idea here for elapsed efforts.


p2 query performance

Our p2 based on installer suffered performance issue when querying IUs from repositories. Though the repositories have a large number of IUs to be queried, but we find the performance of using QL is unacceptable in some special scenarios.

I published several different methods to find the expected IUs. Thomas pointed out the better expression of QL and finally helped us to find out the our repository without IIndexProvider implementation.

IIndexProvider implementation of a repository is quite important to improve the performance of QL, especially use the 'traverse' clause to query something.

And Slicer API is an alternative method when querying the complete dependencies.