EUGENE, Ore. -- KIDSPORTS has taken action against a volunteer coach and referee after a fight during a fifth-grade flag football game at Cal Young Middle School, and the coach is defending his actions.

KIDSPORTS Executive Director Bev Smith said the coach, Phil Olietti, has been relieved from his duties for the season and they removed the referee from his official duties.

Olietti said the action KIDSPORTS took against him was extreme. He claims he never hit the referee.

"I think a game or two suspension would've been more warranted in this situation, given the fact it was not my fault punches were thrown," Olietti said. 

RELATED: KIDSPORTS INVESTIGATING REPORTED FIGHT AT EUGENE GAME

Olietti claims the referee was aggressive and "unstable" during the game. He said when the referee called the game over, he walked over to him arguing that the stop clock rule wasn't being used and the game shouldn't be over. Olietti said the referee yelled at him and asked him to leave the field, but he said he still wanted an explanation and called the referee "fat boy."

"I said I'm not going to get out of your face fat boy. At that point, he started swinging punches at me," Olietti said. "I did not react initially in any way. I think very quickly in that situation, my initial reaction was to punch him back hard. I restrained."

Olietti said he filed a police report after the fight. A spokeswoman for the Eugene Police Department said the case is still open and under investigation. They said no charges have been filed as of Tuesday morning.

Olietti said he has coached youth sports for years and has never been kicked out of a game before.