Random Projections Fail to Preserve Most ELA Features
Empirical study finds linear random projections alter most ELA features computed from identical samples, limiting their reliability for high-dimensional AI optimization despite distance-preservation guarantees.
Random projections via Gaussian embeddings frequently distort geometric and topological structures measured by Exploratory Landscape Analysis when reducing high-dimensional black-box optimization problems (Olarte Rodriguez et al., arXiv:2604.13230). Starting from identical sampled points and objective values, features computed in projected spaces diverged from original-space counterparts across tested sample budgets and embedding dimensions, with only a small subset showing comparative stability.
The Johnson-Lindenstrauss lemma establishes that random projections preserve pairwise distances with high probability in lower dimensions (Johnson and Lindenstrauss, 1984; Dasgupta and Gupta, arXiv:cs/0201002), yet this does not extend to the specific ELA feature classes that quantify multimodality, dispersion, and information content (Mersmann et al., doi:10.1145/2001576.2001690). Prior coverage of dimensionality reduction in AI optimization commonly assumes such projections retain intrinsic landscape properties without empirical verification on ELA metrics.
Applications in large-scale AI systems, including loss-landscape studies for deep neural networks (Li et al., arXiv:1802.06396) and hyperparameter search, increasingly rely on random embeddings to counter the curse of dimensionality; however, the synthesized results indicate that observed robustness can reflect projection artifacts rather than original problem characteristics (Kerschke et al., arXiv:1810.03805).
AXIOM: Random projections preserve distances in theory but distort the majority of ELA features that characterize optimization landscapes, indicating that dimensionality-reduction pipelines common in large-scale AI training and tuning may operate on misleading signals.
Sources (3)
- [1]Does Dimensionality Reduction via Random Projections Preserve Landscape Features?(https://arxiv.org/abs/2604.13230)
- [2]On Random Projections and the Johnson-Lindenstrauss Lemma(https://arxiv.org/abs/cs/0201002)
- [3]Exploratory Landscape Analysis(https://doi.org/10.1145/2001576.2001690)