Recent blog posts by Andrej Karpathy at Medium.com and Pete Warden at PeteWarden.com have caused a paradigm shift in the way I think about neural nets. Instead of thinking of them as powerful machine learning tools, the authors instead suggest that we should think of neural nets, and in particular, convolution deep nets, as ‘self-writing programs.’ Hence the term, “Software 2.0.”
It turns out that a large portion of real-world problems have the property that it is significantly easier to collect the data than to explicitly write the program. A large portion of programmers of tomorrow do not maintain complex software repositories, write intricate programs, or analyze their running times. They collect, clean, manipulate, label, analyze and visualize data that feeds neural networks. — Andrej Karpathy, Medium.com
I found this to be a dramatic reversal in my thinking about these techniques, but it opens up a deeper understanding and is much more intuitive. The fact is that combinations of artificial neurons can be used to model any logical operation. Therefore you can conceptualize training a neural net as searching programming space for an optimal program that behaves in the way you specify. You provide the inputs and desired outputs, and the model searches for the optimal program.
This stands in contrast to the “Software 1.0” paradigm where the programmer uses her skill and experience to conceptualize the right combination of specific instructions to produce the desired behavior. While it seems certain that Software 1.0 and 2.0 will co-exist for a long time, this new way of understanding deep learning is crucial and exciting, in my opinion.
Recently, I’ve been doing some research into Hailey-Hailey Disease (HHD). HHD is an autosomal dominant genetic disorder that leads to severe dermatosis. The disease causing variants are located in the ATP2C1 gene, which is a magnesium-dependent, calcium transporting ATPase.
There are unfortunately few treatment options for HHD. Many treatment options have been tried, from corticosteroids to tacrolimus. There are very few HHD patients, and therefore no large scale clinical trials of therapies for this disease.
I came across a paper that shows that a novel approach, low-dose naltrexone (LDN), may be an effective and low-cost therapy for treating HHD. What is more remarkable, however, is the fact that LDN has already been used with success to treat many diseases like fibromyalgia, Crohn’s disease, and HIV.
Here is the complete list of diseases that LDN has been used to treat with some success according to some case reports and small-scale clinical trials:
Atopic eczema
Cholestatic pruritus
Crohn’s Disease
Adenoid cystic tongue carcinoma
Fibromyalgia
HIV
Multiple Sclerosis
Chronic eczema and pruritis
Hailey-Hailey Disease
******
How is LDN effective across so many seemingly unrelated diseases? I can’t really answer that question. We do know that naltrexone is an opioid receptor inhibitor that is used in the treatment of alcohol and opioid abuse at higher doses. At low dose, the mechanism of action is less clear, but some studies suggest increases in beta endorphins and suppression of cytokines using LDN.
As of now, LDN remains an “off-label” use of naltrexone and in the realm of internet anecdotes until more rigorous studies can be completed. Regardless, it is an exciting development in the potential treatment of rare diseases, like HHD.
Are differentially expressed (DE) genes also phenotypically important?
A new paper in Cell Reports utilizes RNA-seq and Tn-seq (the “tn” in tn-seq stands for transposon) to map the transcriptional and fitness changes in bacterial gene networks in response to stressors, like nutrient depletion and antibiotics.
The transcriptional response measures changes in gene expression as measured by RNA-seq. The fitness or phenotypic response describes the importance of each gene to the response. This is measured by a different assay, Tn-seq, which takes advantage of transposon insertion to selectively inactivate genes in the bacterial genome. Those genes that are depleted in the stressor condition are determined to be “high fitness” (owing to the fact that the bacteria without those genes died under stress).
First, before even considering DE genes, they found that there is no correlation between a gene’s transcriptional abundance (not fold change) and it’s fitness. While most high-fitness genes were also high abundance, many more high-abundance genes were not high-fitness. Thus, there is no useful relationship between a gene’s abundance and fitness.
Superficially, however, one might expect that genes that show large changes in abundance (i.e., large DE) in response to stressors would also be critical for the phenotypic expression of the bacteria’s stress response. That is, those genes with high differential expression would confer high fitness on the cell.
Testing the DE / high fitness relationship
As it turns out, little is actually known about this, and in this paper, Opijnen, et. al., set out to test the idea to determine if in fact high DE genes are also high fitness.
The researchers looked at comparing differential expression in response to a reduced nutrient environment (a type of minimal media) and an antibiotic stress versus high-fitness genes. They found no correlation:
Expression vs. fitness changes for various strains of S. pneumoniae during nutrient depletion (left) and antibiotic stress (right).
You can see from the figure that high fitness genes (those on the far left of the x-axis), are not correlated to high DE genes. There are no genes in the upper-left quadrant of either plot, showing that there is no correlation between fitness and high DE in response to either nutrient or antibiotic stress.
Gene networks co-localize high DE and high fitness genes
Even though the authors found no correlation between DE and fitness changes for individual genes, they took the next step and constructed a metabolic gene network for the S. pneumoniae bacteria. Mapping the DE and fitness changes onto this network revealed a key finding: the high DE genes co-localize in pathways with the high fitness genes. That is, a biochemical pathway might have some members that are high DE, and others that are high fitness. An example of this is the shikimate pathway shown below:
Changes in gene expression and fitness in the Trp biosynthesis pathway. The dashed line is the branch point into Trp synthesis.
The first half of the pathway consists of six genes with significant fitness changes (red boxes) in a row. The next seven genes, from the Trp branchpoint (blue dashed line) are not high-fitness, but do show high DE expression, with four reaching statistical significance. It is not really understood why this happens, but the authors theorize that having the bottom half of the pathway under transcriptional control allows the bacterial to control flux into Trp synthesis and other AA sub-pathways while always maintaining a stable supply of the starting point intermediates (the product of SP1374) through reversible, end product-regulated biosynthesis.
Transcriptomic data should not be used as a surrogate for functional importance
The authors point out that the reliance on trancriptonal abundance changes as markers for functional importance in bacteria, particularly in drug discovery efforts, may be misguided and need to be revisited in light of this and other studies. They also point on that the response to an “orderly” stressor (like nutrient depletion) for which the bacterium is evolved, is likely to be much more clearly defined on a network basis. While the response to a disorderly stressor (a novel antibiotic, for example) may provoke a disorderly transcriptional and fitnress response that can’t easily be interpreted from network analysis. This has important implications for the design of next-generation antibiotics.
