The CSX train, traveling from Selkirk, New York, to Waycross, Georgia, came off the tracks on Water Street around 3:30 p.m.
Two injuries were reported, and officials say they are non-life threatening.
Orange County officials said there are 17 rail cars and two locomotives off the tracks, and electrical lines are down across them.
A small equipment fire broke out at the time of the derailment, but was quickly extinguished.
Newburgh Fire Chief Terry Allers said it's believed there was some kind of a work vehicle close to the rails or on the rails at the time of the derailment. That report is currently being investigated.
Here's a look at the scene as captured by a drone:
HazMat was requested to the scene, as firefighters said there was a minor sulfuric acid leak.
"Some of the freight is hazardous materials, they have some sulfuric acid tankers. We don't believe any major leaks or hazmat releases occurred," said Brendan Casey, Orange County ESU.
CSX confirmed that the train was transporting hazardous materials, but no other leaks were reported. The cleanup and removal of the train won't be easy for CSX.
"I don't think they are going to lift it because it's right underneath power lines, so there is probably a process where they are going to pull it back onto the tracks and move it out from there," Casey said.
Eyewitness News viewer Joni Dunning Armstrong was driving on Water Street and came up on the train in the road.
"Most terrifying thing I've ever seen," she wrote on Facebook. "Sitting at a stoplight and seeing a train, on fire, jump the tracks and start up the road behind you."
Here's a photo she posted:
The train consisted of three locomotives and 77 freight cars, including 38 loads of freight and 39 empty cars. The freight includes sulfuric acid and sodium hydroxide, as well as cardboard, corn oil and glass products.