JRE/JDK's certificate issue and solution

The problem came from I tried to set up send mail server(SMTP) for my Gerrit server. My Gerrit server is using OpenID for user authorization, so I registered a new email account to send notification from Gerrit.

Most of email service providers require the secure authorization when using its SMTP server to send mail. However the root CA of my email provider is not added into the default certificate of JRE. So Gerrit always failed to send email due to ssl verification exception.

My solution is adding the certificate of SMTP server into the certificate used by JRE.

The detail steps are below,

1. Use open_ssl utility to the certificate of SMTP server or its root CA of email provider. Below command can list the certificate of SMTP and its chain. You can paste any of them into a file.
   

      openssl s_client -connect smtp.163.com:465 

2. Then import the certificate saved in previous step into my JRE's key store. The default password of JRE's default keystore is 'changeit'. You can find the cacerts under jre/lib/security folder.
   

    sudo keytool -import -keystore cacerts -alias Smtp163com -file /tmp/smtp.163.PEM 

The tips of Maven/Tycho building crossplatform RCP and repository

I successfully converted our product build from PDE build to Maven/Tycho. Something is worth to be documented here.

There are several examples and posts to demonstrate how using Tycho building your Eclipse plug-ins, features, applications and products. The most helpful example is the demo of Tycho project.

Below are some traps I met when building my project by Tycho,

1. product build
   Our product is based on plug-ins, however we added the 'featurelist' in build.properties of PDE build to include some root binary for the product. However Tycho doesn't support this type of build, we create some features as the placeholder of plug-ins. Then change the product as features based. You have to manually remove the plugins tag in .product definition file, otherwise Tycho will fail on strange error if the .produce has both features and plugins tag. Then configure the director plugin as not installing features.
   

      
       org.eclipse.tycho
       tycho-p2-director-plugin
       ${tycho-version}
       
        
         materialize-products
         
          materialize-products
         
         
          false
          myappprofile
         
        
        
         archive-products
         
          archive-products
         
        
       
      
   

   And I used below way to customize the qualifier string of our build.
   

       
       org.eclipse.tycho
       tycho-packaging-plugin
       ${tycho-version}
       
        '${qualifier-prefix}_'yyyyMMddHHmm
       
         

   
   An limitation of director plugin is that no way using different profile name for the application installed on different hosts. I contributed a patch on bug 362550 for this enhancement.
2. feature build
   We have some features to pack some binary files as root files. But Tycho doesn't support root folder that is recognized by PDE build. The workaround is creating an additional folder, then put the root files into it.
   Meanwhile Tycho doesn't support wildcard to other native touch points, such as changing the files permission. For static file list use comma separated list as workaround.
3. eclipse test plug-in
   I have a plug-in whose scope is 'test', but it doesn't have test case and no dependency for any test framework, such as junit 3.8 or junit 4. And it's used for mocking test server. Configure surefire plugin to let it build as test plug-in as well.
   

    
     
      
       org.eclipse.tycho
       tycho-surefire-plugin
       ${tycho-version}
       
        
         
          junit
          junit
          4.1
         
            
        false
       
      
     
    
    
    
     
      junit
      junit
      4.1
     
    

   And configure the surefire plugin like below to test code in Maven build.
   

    
     
      
       org.eclipse.tycho
       tycho-surefire-plugin
       ${tycho-version}
           
         
        
         my.group
         
          my.feature
         
         ${version}
         eclipse-feature
        
        
         my.group
         
          my.testserver
         
         1.0.0
         eclipse-plugin
        
        
        ${testSuiteName}
        ${testClassName}   
        -Dcom.sun.management.jmxremote
        -consoleLog
        
         
          org.eclipse.equinox.ds
          1
          true
         
        
       
      
     
    
   

4. sign jars
   Add below signjar plugin into parent pom.xml, however I met the md5 error when materializing the repository built on .product. There is a workaround mentioned on Bug 344691.
   

         
         org.apache.maven.plugins
         maven-jarsigner-plugin
         1.2
         
          ${keystore}
          MyCompany
          ${storepass}
          ${keypass}
          true
          ${skip.jar.signing}
          
           -tsa
           https://timestamp.geotrust.com/tsa
          
          
           **/artifacts.jar
           **/content.jar
          
          
           jar
           eclipse-plugin
           eclipse-feature
           eclipse-test-plugin
          
         
         
          
           sign
           
            sign
           
          
          
           verify
           
            verify
           
          
         
        
   

Migration Clearcase to Git -- part 2

Several days ago I had a post to record the unsuccessful experience to migrate source code from Clearcase to Git.

We have a new way after doing some brain storms. This way still is not a perfect solution, but it's much better than previous trial.

1. Use clearexport_ccase to export the source folder to intermittent data. See documentation of Clearcase admin.
2. Create a temporary vob for importing the data later. See example.
3. Import the data into temporary vob. See example.
4. Repeat step 1 to 3 for importing all necessary data into temporary vob.
5. Use the SVN Importer to import the temporary vob as Subversion repository.
6. Last steps refer to a documentation of succeeded migration case of one of Eclipse project from Subversion to Git.

Git definitely is greatest SCM tool now. The size of Subversion repository is around 10GB, finally the Git repository is less than 700MB, which saves more than 10 times disk space. It's awesome!

The flaw of this way is that the removed elements in Clearcase(said using Main/LATEST as cspec of Clearcase vob when exporting) would lose after importing into a temporary vob. So switching to a maintenance branch or tag like 1.0/2.0 in Git, the source code is incomplete. The files existed in that branch or tag, then removed in latest code base are lost. The workaround could be manually checking in GA version to have complete code.

If anybody have graceful and perfect solution to migrate Clearcase to Git, I think he could start a new business. :)

Migrate Clearcase to Git

I tried to migrate the source code of project from Clearcase to Git repository. As far as I know there is no elegant solution for such migration. For purpose of this migration, I want to keep the history and label of files in Clearcase after migrating to Git repository.

There are mature tools to migrate CVS/SVN repository to Git, so I tried to use Subversion as a bridge for my migration.

I used a free software 'SVN Importer' to import the Clearcase vobs to Subversion. The tool is great, and it keeps the history of files, labels and branches. The entire size of new Subversion repository has near 50GB which is unacceptable size of Git repository. The subversion repository contains a lot of legacy code and unwanted binaries, so removing those revisions could significantly reduce the size of subversion repository. And subversion provides some admin tools to manipulate the metadata of subversion, it's possible to remove the unnecessary revisions and re-create a subversion repository with refined content. But I don't have any experience to use the admin tool of subversion before, I failed to filter the unwanted data. It's not worthy of costing too much effects on it. Finally I failed to filter the subversion repository.

Actually the detail history of files is rarely used. If need, we still can find it in Clearcase. At last I manually checked in the released version of our project into Git repository, and tagged them.

Wrote this unsuccessful idea here for elapsed efforts.

p2 query performance

Our p2 based on installer suffered performance issue when querying IUs from repositories. Though the repositories have a large number of IUs to be queried, but we find the performance of using QL is unacceptable in some special scenarios.

I published several different methods to find the expected IUs. Thomas pointed out the better expression of QL and finally helped us to find out the our repository without IIndexProvider implementation.

IIndexProvider implementation of a repository is quite important to improve the performance of QL, especially use the 'traverse' clause to query something.

And Slicer API is an alternative method when querying the complete dependencies.

Create an import library for building application in MinGW

Yesterday I modified an existing c++ application for Windows. And its default build environment is Makefile and MinGW.

However I used a newly Windows API that is not included by header files of MinGW.

First of all, I copied the constant definition from header file of Windows SDK, and defined the Windows API method as a extern C method. So it's no problem to compile the code in MinGW.

Secondly I have to fix the link issue. Because the symbol of the Windows API also can't be found by gcc link.

Here great thanks to Google. It's quite easy to get the knowledge from others.

I found a way to create an library by using dlltool. Dlltool is a utility to create an library with specified methods from existing dll library, which can be used by gcc link later.

Below are links I referred to create an import library,

[1] http://www.emmestech.com/moron_guides/moron1.html
[2] http://www.mingw.org/wiki/CreateImportLibraries
[3] http://lists-archives.org/mingw-users/19461-import-library-for-c.html

Customize PDE build

The documentation of PDE has a chapter for this topic. Basically it's simply. Copy the template scripts what you want from templates/headless-build folder under org.eclipse.pde.build plug-in to your build configuration directory that is the folder has build.properties file.

However I found the variables listed in template 'customAssembly.xml' can't be used in the runtime. I filed bug 346370 against it.

Using the certificate of Windows code signing to sign jars

I did sign the jars via reusing the existing certificate of Windows code signing several months ago. Writing it down for further reference.

Whatever your purpose of reusing the existing Windows code certificate, I only document the way from technical perspective.

After buying the certificate of Windows code signing from CA, you will get a .pvk file that stores both the certificate and private key. PVK file is the PKCS12 format[1], however java uses JKS format by default. So you need convert the pvk file to JKS keystore and certificate.


Since 6.0 JDK supports PKCS12 directly, you can use 'jarsigner' and PVK file to sign jars directly[2].

jarsigner -keystore /working/mystore.pvk -storetype pkcs12 -storepass myspass -keypass j638klm -signedjar sbundle.jar bundle.jar jane

Or using keytool to convert the PKCS#12 to JKS format[3] if using Eclipse PDE build to sign your jars.

keytool -importkeystore -srckeystore KEYSTORE.pvk -destkeystore KEYSTORE.jks  -srcstoretype PKCS12 -deststoretype JKS -srcstorepass mysecret -deststorepass mysecret -srcalias myalias -destalias myalias -srckeypass mykeypass -destkeypass mykeypass -noprompt 
[1] http://en.wikipedia.org/wiki/PKCS
[2] http://download.oracle.com/javase/6/docs/technotes/tools/solaris/jarsigner.html
[3] http://shib.kuleuven.be/docs/ssl_commands.shtml#keytool

Unlock the locked profile if firefox/thunderbird crash

I met that firefox/thunderbird complained another its instance running even if no a running firefox/thunderbird process. Finally let them run again after removing the '.parentlock' file in their default profile.

strace utility helps me a lot to find the solution.

strace -f -e file firfox

Eclipse P2's import/export capability

I implemented the replication tool at the end of 2009, then published it to Eclipse Marketplace in May 2010. However it's not pervasively used due to users have to install that plug-in firstly.

I searched a similar request on bugzilla, then I initialized my contribution in the early of this year. Finally it was accepted and will release as part of eclipse itself since Eclipse 3.7 M7! I hope it would benefit the users of Eclipse more and more.

And I was nominated and elected as the committer of Equinox p2, it's a great honor for me. :)

[vim] delete the lines not contain words

:g!/some expression/d