Ja’Kye Joseph, 6, and Branario Minto, 5, crept out of the flat they lived in in North Lauderdale, Florida while their mother, Wildine Joseph, was asleep.
The brothers managed to leave the flat and skip a locked gate that the pool is surrounded by.
CBS Miami reported that a neighbour saw the boys facedown in the pool at 9.30pm while Joseph broke down shortly after, screaming: “I’m sorry, I was sleeping and didn’t get there fast enough!”
Ms Joseph said that the children’s father attempted CPR on one of the boys while she pulled the other from the pool.
In a heartbreaking interview given to local news station Local 10, Ms Jones tearfully said: “I’m so sorry. If I was there I could have saved them on time.
“I will always miss my babies. I’m sorry, Ja’Kye. I’m sorry, Branario.”
Ms Joseph also said her oldest son, Ja’ Kye, was taking swimming lessons.
CBS Miami also reported that Ms Joseph said: “I wasn’t there to save them but they’re in a better place now. That’s all that matters.
“I will always miss my babies. I’m sorry Jakye, I’m sorry Benario. They were only five and six, I lost my babies too early.”
Neighbour Marie Flemming said she heard a commotion on Wednesday night and ran out to unlock the pool gate before calling 911.
She said: “To find out they both died, is really hitting me hard.”
At the time, Broward Sheriff’s Office said a four-month-old and a one-year-old have been taken out of Joseph’s care by social sevices.
On Friday a judge ruled that the younger siblings of the two boys will remain in state custody.