Unix one-liner to convert VCF to Oncotator format

Here is a handy unix one-liner to process mutect2 output VCF files into the 5 column, tab-separated format required by Oncotator for input (Oncotator is a web-based application that annotates human genomic point mutations and indels with transcripts and consequences). The output of Oncotator is a MAF-formatted file that is compatible with MutSigCV.

#!/bin/bash
FILES='*.vcf.gz'
for file in $FILES
do
zcat $file | grep -v "GL000*" | grep -v "FILTER" | grep "PASS" | cut -d$'\t' -f 1-5 | awk '$3=$2' | awk '$1="chr"$1' > $file.tsv
done

Breaking this down we have:

“zcat $file” :  read to stdout each line of a gzipped file

“grep -v “GL000*” :  exclude any variant that doesn’t map to a  named chromosome

“grep -v “FILTER” : exclude filter header lines

“grep “PASS””:  include all lines that pass mutect2 filters

“cut -d$’\t’ -f 1-5”  : cut on tabs and keep fields one through five

“awk ‘$3=$2’ :  set column 3 equal to column 2, i.e., start and end position are equal

“awk $1=’chr’$1″” : set column one equal to ‘chr’ plus column one (make 1 = chr1)

 

Automate your Topspin NMR workflow

Here is a tip for scientists that need to batch process NMR data quickly and uniformly for analysis.  This approach could be a big time-saver in situations where you have a large series of 1D reference spectra collected by sample automation, for example.  Or in NMR screening applications, where dozens of STD-NMR experiments are being collected during an overnight run.

Hidden away in the Topsin “Processing” menu is a feature called “Serial Processing:”

Screen shot 2014-10-20 at 2.39.51 PMSelect this menu option and you will see the following dialogue:

Screen shot 2014-10-20 at 2.49.37 PMSince this is first time you are doing this operation, you need to select “find datasets” in order to first find the data to process.  In the future, you will have a “list” created for you by the program that you can reuse to reference datasets in combinations that you specify.

When you click “find datasets” you will see this dialogue:

Screen shot 2014-10-20 at 2.40.02 PMSelect the data directory to search from the “data directories” box at the bottom of the window.   (If your NMR data directory is not here, it is because you haven’t added it to the Topspin file brower in the main Topspin window.  Go do that first, and come back and try this operation again.)

Under the “name” field, enter the name of the specific dataset directory you wish to search, or leave it blank to search across many directories.  You can also match on experiment number (EXPNO) or process number (PROCNO).  The check boxes enforce exact matching.  You can select 1D or higher dimensional datasets for processing.  You can also match by date.

When you’ve made your selections, it will look like this:

Screen shot 2014-10-20 at 2.40.46 PMIn this search, I am selecting for all 1D data contained in the “Oct16-2014-p97” subdirectory of my NMR data repository at “/Users/sandro/UCSF/p97_hit2lead/nmr”.

Click “OK” and wait for the results.  Mine look like the following:

Screen shot 2014-10-20 at 2.41.18 PMThe program has found 24 datasets that match my criteria.  At this point, you want to select only those you wish to batch process.  I will select all files like this:

Screen shot 2014-10-20 at 2.41.36 PMNow click “OK” and you are returned to this prompt:

Screen shot 2014-10-20 at 2.42.00 PMNotice that the program has now created a list of datasets for batch processing for you, store in the ‘/var/folders/’ temporary directory.  The list is a text-based list of the filenames you specified by your selection criteria.  You can edit by hand or proceed to the next step.  To proceed, click “next.”  You will now see this dialogue:

Screen shot 2014-10-20 at 2.42.20 PMThis is where the useful, time-saving stuff happens.   This dialogue takes the list you defined and applies whatever custom command sequence you would like to apply to your data.  You define this sequence in the text box at the bottom.  As you can see, I have chosen to perform “lb 1; em; ft; pk.”   This is line broadening = 1, exponential multiplication, fourier transform, and phase correction.  You can also specify a path to a python script for the Topspin API.

Once you have your desired processing commands, click “Execute” and go grab a coffee!  You just saved yourself many minutes of routine processing of NMR spectra.    Hope you find this tip useful and that it can save you some time in your day.