Lord Frost told Conservative supporters he is "extremely concerned" that Brexit could be under threat after the next general election.
The conservative peer, who negotiated Boris Johnson's original exit deal and trade agreement with the EU, said he feared Labour would move quickly leave the UK more economically tied to Brussels.
And he lay blame in part at the current Government as he urged ministers to start championing the successes of Brexit and not let negativity fill the headlines.
"A lot has been done, and we must keep out of the gloom," he told a Centre for Brexit Studies event on the sidelines of the Conservative Party's annual conference.
WATCH: Lord Frost's three demands from Sunak
He told those gathered in Manchester: "I'm very concerned about what Labour would do if they ere to win in 12 months time or whenever.
"Whatever they say, I could see they are keen to take us back in, at least economically, as quickly as they can and they seem quick happy to get there by one small agreement after another, salami slicing, making the case each time.
"So, I guess as one of the architects of the Brexit we have, I'm extremely concerned and I do want to see us win the next election."
Labour has denied suggestions it wants to tie the UK to the EU but says it wants to work closer with the bloc to "make Brexit work".
Rishi Sunak will address Conservative members on Wednesday
PA
Speaking at an earlier event organised by the Tax Payer's Alliance, Frost set out his vision of what Prime Minister Rishi Sunak could do to help keep the Conservatives in Government.
He set out three points that he wanted to hear in the party leader's speech to the Tory faithful on Wednesday lunchtime.
He said: "Please paint us a picture of how Conservative Britain - if we win again - is different from Labour Britain. What is your vision of society? You know, how should it work? What's the role of government? What's the role of the individual? What is Conservative Britain about?
"You always knew that to Mrs Thatcher what she wanted to create and we're not getting that."
He added: "I would like us to commit to a tax cut before the next election and spending cuts that go with it. I think it's an important piece of credibility to the overall offer.
"And 3rd, I would like a proper sort of red team fundamental look at the net zero project.
"The moment is captured by believers in governments and nobody can tell whether we're doing the right things or not.
"We need a fundamental room branch. Look at that by people who we can trust."
Over the weekend, Sunak was handed a polling boost when one survey suggested he had closed the voting intention gap on Labour by five points.
Despite the positive news, Sunak has faced a party rebellion over the UK's high taxes and has risked seeing the conference overshadowed by a row on the future of HS2.
