Her Majesty reportedly made the comments at a private event.
It is one of the harshest political statements made by the Queen in her 67-year reign.
The 93-year-old has largely kept quiet about her political views during her long time on the throne.
It comes as Britain’s ongoing crisis over Brexit threatens to drag the monarch into the row over the UK leaving the European Union.
Labour’s John McDonnell declared earlier this week that Jeremy Corbyn should go to Buckingham Palace and say “we’re taking over” if Tory PM Boris Johnson loses a confidence vote.
“I think she’s really dismayed”
Johnson has vowed that Britain will leave the EU on October 31 with or without a deal.
This could trigger a second independence referendum in Scotland, and turmoil in Northern Ireland over the border with the Republic of Ireland, potentially tearing the Queen’s Kingdom apart.
Her Majesty reportedly hit out at the UK’s political class and its “inability to govern”.
The Sunday Times reported that the Queen made the exasperated comments shortly after David Cameron’s resignation following the Brexit referendum in 2016, but a royal source said her frustration has since grown.
“I think she’s really dismayed,” they said.
“I’ve heard her talking about her disappointment in the current political class and its inability to govern correctly.”
Describing the Queen’s comments at the time, a senior royal source said: “She expressed her exasperation and frustration about the quality of our political leadership, and that frustration will only have grown.”
Senior civil servants are desperate to keep the Queen out of any political decisions, which could threaten her status in a constitutional monarchy where she has largely ceremonial powers.
Her aides have said in private that parliament must decide who can form a government in the event that Johnson loses a confidence vote.
It comes as speculation grows that Labour will call a vote of no confidence in Johnson when parliament returns early in September.
But there are fears that Johnson could stay on even if he loses and call a general election after October 31, leaving MPs powerless to stop the UK crashing out of the EU without a deal.
Buckingham Palace has not commented on the potential of a constitutional crisis.