Neural Architecture Search has emerged as a transformative paradigm for automating the design of deep neural networks, reducing the reliance on human expertise and enabling the discovery of architectures that often surpass manually designed counterparts. This paper presents a comprehensive survey of NAS methodologies, tracing the evolution from early reinforcement learning-based and evolutionary approaches to modern differentiable and one-shot methods. The paper systematically categorizes NAS techniques along three fundamental dimensions: search space design, search strategy, and performance estimation. A unified mathematical framework is introduced that encompasses gradient-based, reinforcement learning-based, and evolutionary NAS under a common optimization objective.

Contrastive learning has become the dominant paradigm for aligning heterogeneous modalities in multimodal large language models (MLLMs). Information Noise Contrastive Estimation (InfoNCE) Loss and its variants serve as the foundational training objectives for such models. As models scale to billions of parameters and training batches reach millions of samples, the limitations of conventional softmax-based contrastive losses have become increasingly apparent: quadratic memory growth, batch size sensitivity, semantic noise in negative sampling, and the inherent tension between alignment and uniformity. This paper presents a comprehensive survey of the evolution from InfoNCE to next-generation objectives for multimodal LLMs. We analyze the theoretical foundations of contrastive learning through the alignment-uniformity framework, examine the practical innovations that have enabled unprecedented batch scaling, and evaluate emerging alternatives including sigmoid-based losses, non-contrastive alignment methods, and hybrid generative-discriminative objectives.

Large language models (LLMs) have achieved remarkable performance across diverse tasks, yet they remain fundamentally static: once trained, their knowledge is frozen, unable to adapt to evolving information, emerging domains, or shifting user needs without expensive retraining. This limitation - catastrophic forgetting - has become a critical bottleneck as LLMs transition from research artifacts to living systems deployed in dynamic real-world environments. This paper presents a comprehensive survey and unified framework for continuously learning LLMs (CL-LLMs), organizing the field into five complementary paradigms: replay-based methods, regularization-based approaches, parameter-efficient adaptation, architectural isolation, and inference-time continual learning.