On the Loose Use and Methodological Boundaries of the "Manifold" Concept in Current AI

In current AI and deep learning theory research, the term “manifold” is frequently used to describe data distributions, representation spaces, or constraint sets. This is not necessarily an exaggeration of theoretical depth; often it merely borrows geometric intuition or serves as a convenient name. But unless one specifies whether the “manifold” in question is a rigorous mathematical object, a local approximation, or simply a set satisfying constraints, the discussion easily loses precision and its methodological payoff stays quite limited.

Methodological Issues

In interdisciplinary research, the fundamental motivation for importing concepts from a particular branch of mathematics is to invoke the mature tools and theorems established in that field and thereby derive new conclusions. For example, once a specific machine learning problem is placed within the framework of information theory, derivations can proceed with information divergences and the various information inequalities; once the problem is cast as a statistical one, concentration inequalities become the core tool for analyzing generalization error bounds; and once it is abstracted into a convex optimization problem, researchers can draw on the systematic toolkit of convex analysis to prove convergence and convergence rates of algorithms.

In current AI research, however, “manifold” is often used only as a conceptual label. Many papers are content to claim that “data lies on a low-dimensional manifold” or that “some class of constrained objects lies on a manifold,” without going on to use the core tools of differential geometry, such as curvature, geodesics, local coordinate changes, or integration and measure structures on manifolds. This does not mean such work lacks value; it means that “manifold” here is often closer to modeling language than to a mathematical framework capable of yielding new theorems. When a concept is imported without its accompanying toolkit, its explanatory power should be assessed with care.

Local Properties and Global Structure

Tian Yuandong has criticized this phenomenon in an interview. Take the “pothole manifold hypothesis” put forward to explain adversarial examples: unless the topological structure, regularity conditions, embedding, or computable geometric quantities of the manifold are specified, the mathematical explanatory power of such a claim is limited.

In high-dimensional spaces, many complex data distributions can, in a loose sense, be described as low-dimensional structures, locally smooth objects, or subsets embedded in a high-dimensional space. But claiming “this is a manifold” without specifying its topological structure or geometric features conveys very little. Intrinsic dimension, local smoothness, and neighborhood approximability are mostly local properties, and they need not suffice to characterize the global structure relevant to generalization, robustness, or optimization dynamics. The issue is not that the term “manifold” cannot be used, but that the name alone can hardly carry the explanatory burden.

An Example

Take DeepSeek’s mHC: Manifold-Constrained Hyper-Connections as an example. The “manifold” there is closer to a name drawn from engineering practice or geometric intuition, and it does not necessarily correspond to Riemannian manifold optimization in the rigorous sense.

If the core operation merely treats a set $\mathcal{S}$ satisfying specific constraints as a manifold $\mathcal{M}$ and performs restricted updates on it, then in terms of mathematical content it looks more like a constrained optimization problem. Calling such an object a manifold is not necessarily wrong; under suitable regularity conditions, a constraint set may indeed be a manifold. But if the paper does not go on to use Riemannian metrics, tangent space projections, geodesics, or other key structures of manifold optimization, then the more accurate statement is that the authors adopted geometric terminology rather than establishing a rigorous theory of manifold optimization. What deserves discussion is not whether the name sounds “advanced,” but whether it corresponds to additional mathematical content.

Conclusion

Scientific research should avoid letting terminology carry theoretical commitments beyond what the argument can support. If the concept of “manifold” is used in AI theory, one should state whether what is meant is the rigorous differential-geometric definition, a local approximation, a modeling intuition, or an engineering shorthand. If the work is essentially about a constrained optimization problem or a low-dimensional structure assumption, using that language directly is usually more accurate than a vague appeal to “manifold.” Progress in theoretical machine learning ultimately rests on testable definitions, clear assumptions, and solid mathematical derivation.

Citation

If you need to cite this article, please refer to:

@article{zou2026manifold,
  title={论当前 AI 界内“流形”概念使用的泛化与方法论边界},
  author={Zou, Jiaxuan},
  journal={Jiaxuan's Blog},
  year={2026},
  url={https://jiaxuanzou0714.github.io/blog/2026/manifold/}
}