He made the promise as the government announced £10m to improve security and conditions at certain jails.
Prisons minister Rory Stewart: I'll resign if drugs and violence don't go down
New body scanners and sniffer dogs are to be introduced in 10 of England's most "challenging" prisons as part of a clampdown on drugs and mobile phones.
The dogs have been trained to detect psychoactive substances such as Spice.
Ministers also want to raise standards of leadership by sending prison governors to military-style colleges.
And there will be a programme of repairs and improvements to cell windows and perimeter security in the prisons.
The measures are being targeted at 10 jails which the Ministry of Justice says have "acute problems" with high drug use, violence and building issues.
Hull - Category B
Humber, East Riding of Yorkshire - Category C
Leeds - Category B
Lindholme, near Doncaster - Category C/D
Moorland, Hatfield Woodhouse, South Yorkshire - Category C
Wealstun, near Wetherby, West Yorkshire - Category C
Nottingham - Category B
Ranby, Nottinghamshire - Category C
Wormwood Scrubs, west London - Category B
Isis, south-east London - Category C
It comes after figures published by the Ministry of Justice last month showed incidents of self-harm and assaults in prisons were at record levels. Seizures of smuggled-in drugs and mobile phones also rose by 23% and 15% respectively in the year to March.
Mr Stewart said the changes would pave the way for a "new ethos and new direction" for prisons, saying: "It is vital we set challenging standards so prisons are places where offenders can turn their lives around.
"We need to make these prisons calmer, more orderly places and in the end that comes down to challenging and managing prisoners consistently, firmly and fairly."
By Danny Shaw, BBC home affairs correspondent
While much of Whitehall is on holiday, the Ministry of Justice has been busy - taking advantage of the sleepy summer news agenda with a series of press releases about prison reform.
The announcements have included a report on sport in prison, a pilot scheme to promote community sentences and plans for in-cell telephones.
The latest measures - £10m for the 10 most challenging prisons - will certainly make a difference. X-ray scanners, for example, are highly effective at detecting drugs - it's just a wonder it's taken so long for them to be introduced.
The energetic prisons minister Rory Stewart, who has immersed himself in the brief since he was appointed, is right to point out that the key to a stable prison is leadership.
But his idea to raise standards among governors, with a military-style training college, will take time to yield results.
Of the £10m investment, £6m is being earmarked for security measures, £3m for the repairs and £1m for training.
The Ministry of Defence has been contacted for help in setting up a training scheme for prison governors, similar to that for senior members of the armed forces.
However, the Prison Reform Trust said the problems in the jails were caused by a failure to match resources to demand.
Its director, Peter Dawson, said: "It was a catastrophic failure to provide that balance which caused the collapse of prison safety after 2012 - trying to tell governors how to run prisons is not going to put it right."
Shadow justice secretary Richard Burgon said the government must go "much further" and set out an emergency plan across the prisons estate with substantial new funding to put an end to this crisis and make our prisons safe and humane for staff and inmates".
There are about 84,000 inmates in more than 100 prisons, across England and Wales.
Read more news of London on our site.