Athletic director Jeff Hammond announced the decision on Tuesday morning, three days after the miserable 0-12 season ended with a loss to Memphis on Saturday.
"At Southern Miss we expect to compete hard and succeed both in the classroom and on the field," Hammond said in a statement. "This is not the exception.This is in fact the standard, the norm, our identity and who we are."
The decision wasn't unexpected, though it will cost the Golden Eagles. The 60-year-old Johnson is due a $2.1 million buyout over the next three seasons.
The veteran coach had been a successful defensive coordinator at several schools, including South Carolina, Mississippi State, Alabama and Clemson before his arrival in Hattiesburg, Miss.
He was also the defensive coordinator at Southern Miss in the late 1980s and was hailed as an experienced coach who had good ties to recruiting in the deep South when he was hired.
His experience didn't translate into head coaching success with the Golden Eagles — and the program suffered a stunning fall under his leadership.
Hammond gathered players for a meeting on campus Tuesday afternoon. Southern Miss linebacker Alan Howze said the AD explained the decision to fire Johnson and asked the players to continue to work hard during the search for a new coach.
"Personally, I thought we were going to get a second chance," Howze said. "I'm not going to say I never saw (Johnson) leaving, but I thought he was going to come in here again. We were going to work as a unit again and prove ourselves as a team."
Southern Miss finished without a victory one season after a school-record 12-2 campaign that included a Conference USA championship and a win in the Hawaii Bowl. The Golden Eagles were the only team in the Football Bowl Subdivision not to win a game this year.
It is the first time Southern Miss has had a losing record since 1993 and the first time the Golden Eagles have missed the postseason since 2001.
Southern Miss fullback Bruce Johnson said the team is ready for a return to winning.
"Going through this as a 21-year-old guy, I'm going to look back on it one day and be thankful," Johnson said. "I know that's not what the fans want to hear or what the people who give this program money want to hear. But we're growing as men around here. We went from 12-2 to 0-12 and we can go to 14-0 if we want to. We can make this whatever we want it to be."
Hammond said the search for Johnson's replacement has already started.
"Today marks a new beginning, a new season and a new start," Hammond said.
Johnson took over the program in December after Larry Fedora left for North Carolina. He inherited a roster that had 13 returning starters, but could never find consistency on offense or defense.
Several factors played into Southern Miss' awful season, including a difficult early schedule that included losses to Nebraska, Louisville and Boise State.
The Golden Eagles also fell short in close games. Southern Miss lost 38-31 in double overtime to Central Florida on Oct. 13.
The quarterback position was also an issue for Johnson. He used five different quarterbacks this season, including four different starters. None were particularly effective, and the Golden Eagles ranked 107th out of 120 FBS teams in passing offense.
Follow David Brandt on Twitter: www.twitter.com/davidbrandtAP