Niche DE
Posted on
cell’s niche: the cell-type composition of its local neighborhood, where the range of the local neighborhood is determined by a kernal function.
Niche-DE identifies cell-type-specific niche-associated genes, defined as genes whose expression within specific cell types is significantly up- or down- regulated in the context of specific spatial niches, as compared to their cell-type-specific mean expression.
- $Y_{c,g}$: gene $g$ in cell $c$
- $T_c$ type of cell $c$
- $\mu_{t,g}$ gene $g$ in a cell of type $t$
- spatial neighborhood: $N_{\sigma, c, t}$
- $N_{\sigma, c} = (N_{\sigma, c, 1}, \ldots, N_{\sigma, c, T})$: the effective niche cell-type composition of cell $c$ with kernel bandwidth $\sigma$.
characterize the spatial niche of a cell as a $T$ dimensional vector where index $j$ measures how much of cell type $j$ is in the neighborhood of the index cell
\[\log E[Y_{c, g}\mid T_c, N_{\sigma, c}] = \log \mu_{T_c, g} + \sum_{n=1}^T N_{\sigma, c, n}\beta_{\sigma, t_c, n}^g\]- index cell type: $i \in {1,\ldots, T}$ (??? no symbol $i$)
- niche cell type: $n \in {1,\ldots, T}$
The goal is to estimate the cell-type-specific niche-differential expression parameters ${\beta_{\sigma, i, n}^g}$
and to identify those genes $g$ and index-niche cell-type combinations $(i, n)$ where $\beta_{\sigma, i, n}^g \neq 0$.
A significant test against the null hypothesis $\beta_{\sigma, i, n}^g = 0$ means that, when the index cell is of type $i$, the enrichment of cell type $n$ in the effective niche is associated with a significant change in the expression of gene $g$ within the index cell.
call gene $g$ an $(i, n)^+$ niche-gene if the association is positive and an $(i, n)^-$ niche gene if the association is negative.