Abstract

Searches are performed for nonresonant and resonant di-Higgs boson production in the b (b) over bar gamma gamma final state. The dataset used corresponds to an integrated luminosity of 139 fb(-1) of proton-proton collisions at a center-of- mass energy of 13 TeV recorded by the ATLAS detector at the CERN Large Hadron Collider. No excess above the expected background is found and upper limits on the di-Higgs boson production cross sections are set. A 95% confidence-level upper limit of 4.2 times the cross section predicted by the Standard Model is set on pp -> HH nonresonant production, where the expected limit is 5.7 times the Standard Model predicted value. The expected constraints are obtained for a background hypothesis excluding pp -> HH production. The observed (expected) constraints on the Higgs boson trilinear coupling modifier kappa(lambda) are determined to be [-1.5, 6.7] ([-2.4, 7.7]) at 95% confidence level, where the expected constraints on kappa(lambda) are obtained excluding pp -> HH production from the background hypothesis. For resonant production of a new hypothetical scalar particle X (X -> HH -> b (b) over bar gamma gamma), limits on the cross section for pp -> X -> HH are presented in the narrow-width approximation as a function of m x in the range 251 GeV = m(X) = 1000 GeV. The observed (expected) limits on the cross section for pp ->> X -> HH range from 640 fb to 44 fb (391 fb to 46 fb) over the considered mass range.