Monthly Archives: June 2012

ssh tunnel

Posted on June 14, 2012 by Xiaomeng (Shawn) Wan

ssh tunnel 1. generate ssh-key and upload public key to server, confirm passwordly ssh 2. ssh -f shawn@xxx.com -L 11111:xxx.com:222222 -N 3. connect to localhost:11111 ssh passwordless ssh-keygen ssh-copy-id shawn@xxx.xxx.xxx.xxx

Posted in Uncategorized | Tagged | Leave a comment

Git commands

Posted on June 7, 2012 by Xiaomeng (Shawn) Wan

# commit all updates (not add new) git commit -a -m”commit message” #pull and overwrite local uncommitted changes git fetch –all git reset –hard origin/master

Posted in git | Tagged | Leave a comment

Maven specify compiler version

Posted on June 5, 2012 by Xiaomeng (Shawn) Wan

if you want to enable assertions/annotations/generics/for-each (-source 1.6) and also want the compiled classes to be compatible with JVM 1.6 (-target 1.6) <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>2.5.1</version> <configuration> <source>1.6</source> <target>1.6</target> </configuration> </plugin> if you want to generate a jar with dependencies … Continue reading →

Posted in maven | Tagged , | Leave a comment

MongoDB count distinct substrings of values

Posted on June 4, 2012 by Xiaomeng (Shawn) Wan

counting distinct values is easy as: db.my_collection.distinct(_id) but counting substrings of the values is not var cnt=new Object() db.my_collection.find({_id:{$exists:true}}).forEach(function(doc){if(doc._id != null && typeof doc._id == “string”) cnt[doc._id.substring(0,doc._id.indexOf(‘:’))]=1;}) var size=0; for(key in cnt) {if (cnt.hasOwnProperty(key)) size++;};

Posted in mongodb | Tagged | Leave a comment

Mahout 0.7 random forest examples

Posted on June 1, 2012 by Xiaomeng (Shawn) Wan

#breiman example hadoop jar $MAHOUT_HOME/core/target/mahout-core-0.7-SNAPSHOT-job.jar org.apache.mahout.classifier.df.tools.Describe -p testdata/glass.data -f testdata/glass.info -d I 9 N L hadoop jar $MAHOUT_HOME/examples/target/mahout-examples-0.7-SNAPSHOT-job.jar org.apache.mahout.classifier.df.BreimanExample -d testdata/glass.data -ds testdata/glass.info -i 10 -t 100 #partial implementation example hadoop jar $MAHOUT_HOME/core/target/mahout-core-0.7-SNAPSHOT-job.jar org.apache.mahout.classifier.df.tools.Describe -p testdata/KDDTrain+.arff -f testdata/KDDTrain+.info -d N … Continue reading →

Posted in mahout | Tagged , | 4 Comments