Abstract

Open Radio Access Networks redefine traditional RAN by introducing functional disaggregation and open interfaces that foster interoperability and innovation, reduce vendor lock-in, and enable flexible 5G and beyond deployments. However, this architectural openness also enlarges the attack surface, creating new vectors of compromise that existing 3GPP protection models do not fully address. Although O-RAN Alliance Working Group 11 (WG11) prescribes standardized security protocols (such as TLS, IPsec, MACsec, and OAuth), their practical implications across heterogeneous interfaces and their underlying cryptographic efficiency remain poorly understood. This paper employs a systematic study methodology that thoroughly evaluates and analyzes the security challenges and research findings related to O-RAN, synthesizing the state of knowledge across architectural layers, protocols, and cryptographic primitives. Our comprehensive and homogeneous evaluation exposes research gaps, biases, and underexplored aspects in O-RAN security, particularly with regard to cryptographic performance and hardware acceleration feasibility. Finally, we present a detailed discussion of open issues and propose targeted future directions to strengthen O-RAN's security posture and cryptographic resilience in evolving deployments.